CMGait: Enhancing Cross-Modality Gait Recognition between LiDAR and RGB through Contrastive Identity-consistent Feature Aggregation

Abstract

Combination usage of LiDAR and RGB cameras for gait recognition can achieve cross space recognition and privacy protection. In addition, the widespread application of LiDAR cameras with 3D geometry information and the large amount of RGB gaits has led to the demand for cross-modality gait recognition on LiDAR and RGB modalities. To address the challenge of cross-modality recognition, we proposed a novel cross-modality gait recognition paradigm called CMGait. The key innovations include a novel projection method for transforming LiDAR point clouds into depth maps, feature alignment modules, Transformer-based identity encoders, and an embedding distance fusion method with similarity matrices based contrastive learning. Experimental results showcase state-of-the-art performance with Rank-1 accuracy of 62.8% and 66.1% for different directions in cross-modality gait recognition. Ablation experiments validate the effectiveness of the proposed methods, highlighting advancements in feature alignment and modality fusion techniques.