Intelligent Signal Processing Mechanisms for Nuanced Anomaly Detection in Action Audio-Visual Data Streams
Josef Kittler, Ioannis Kaloskampis, Cemre Zor, Yang Xu, Yulia Hicks, Wenwu Wang
Abstract
We consider the problem of anomaly detection in an audiovisual analysis system designed to interpret sequences of actions from visual and audio cues. The scene activity recognition is based on a generative framework, with a high-level inference model for contextual recognition of sequences of actions. The system is endowed with anomaly detection mechanisms, which facilitate differentiation of various types of anomalies. This is accomplished using intelligence provided by a classifier incongruence detector, classifier confidence module and data quality assessment system, in addition to the classical outlier detection module. The paper focuses on one of the mechanisms, the classifier incongruence detector, the purpose of which is to flag situations when the video and audio modalities disagree in action interpretation. We demonstrate the merit of using the Delta divergence measure for this purpose. We show that this measure significantly enhances the incongruence detection rate in the Human Action Manipulation complex activity recognition data set.
BibTeX
@inproceedings{icassp2018_intelligentsigna,
title = {Intelligent Signal Processing Mechanisms for Nuanced Anomaly Detection in Action Audio-Visual Data Streams},
author = {Josef Kittler and Ioannis Kaloskampis and Cemre Zor and Yang Xu and Yulia Hicks and Wenwu Wang},
booktitle = {ICASSP 2018},
year = {2018}
}