Sequence-Based Device-Free Gesture Recognition Framework for Multi-Channel Acoustic Signals

Abstract

Device-free gesture recognition schemes based on acoustic sensing signals are promising solutions for next-generation human-computer interaction systems. However, existing gesture recognition frameworks reuse visual neural networks to perform feature extraction. These approaches ignore the time sequence nature of the acoustic signal and treat acoustic echo profiles solely as 2D images. In this paper, we propose a time-sequence-based deep learning framework that can exploit the spatio-temporal information of sensing signals. The framework first fuses multi-channel acoustic signals to extract spatial gesture features from a single acoustic frame and then uses the Transformer network to discover the timing relations between spatial features. Our extensive evaluations with real-world datasets show that our light-weighted framework outperforms the state-of-the-art in classifying 14 gestures and achieves an average accuracy of 95.85%.