mmHIU: a human-to-human interaction understanding system based on mmWave sensing

Abstract

Human-to-human interaction understanding(HIU) plays a significant role in both physical and mental health of individuals in their daily lives. Nowadays, many of HIU tasks are based on visual information, which can compromise individuals’ privacy in daily life. In this paper, we propose mmHIU, a privacy-preserving HIU system based on mmWave sensing. To achieve mmHIU, we need to address two challenges: Extraction of interaction features and processing of raw data in a manner that preserves interaction features. We design multi-level feature extraction network mmHIU-STNet, and clustering-based frame normalization strategy to address these two challenges. We evaluate mmHIU on 10,811 point cloud sequences from 9 pairs of inter-actors, and results show that its accuracy reaches 90.56%, outperforming baseline methods, with a significant improvement in recognizing complex interaction behaviors.