Ultrasound based gesture recognition

Abstract

In this study, we explore the possibility of recognizing hand gestures using ultrasonic depth imaging. The ultrasonic device consists of a single piezoelectric transducer and an 8 - element microphone array. Using carefully designed transmit pulse, and a combination of beamforming, matched filtering, and cross-correlation methods, we construct ultrasound images with depth and intensity pixels. Thereafter, we use a combined Convolutional (CNN) and Long Short-Term Memory (LSTM) network to recognize gestures from the ultrasound images. We report gesture recognition accuracies in the range 64.5-96.9%, based on the number of gestures to be recognized, and show that ultrasound sensors have the potential to become low power, low computation, and low cost alternatives to existing optical sensors.