A Deep-narma Filter for Unusual Behavior Detection from Visual, Thermal and Wireless Signals

Abstract

Detection of unusual behavior is an important topic in signal and image processing. Because of the topic's complexity, addressing it as a solely RGB video analysis problem raises significant challenges. This has resulted in approaches that aim at exploiting different data modalities that can overcome the inherent restrictions of unimodal techniques. Moreover, the classification outcome of such approaches is affected not only by the input data, but also by previous classification history. To this end, this paper introduces a novel deep-NARMA filter that extends a typical CNN architecture, and endows it with autoregressive moving average behavior. In addition, it incorporates a data fusion framework that supplements RGB video streams, with thermal capturing and information about the distortion of WiFi signal reflectance. Experimental results indicate a better performance compared to conventional as well as deep learning approaches.