Rgb-D Based Multi-Modal Deep Learning for Face Identification

Abstract

In recent years, the rapid development of depth cameras and wide application scenarios. The depth image information becomes more influential in face identification. In the proposed architecture, we implement the networks in dual CNN paths for color and depth images separately. Moreover, we design innovative loss functions to strengthen the discrimination and the complementary features between color and depth modalities. To preserve the strengthened color and depth features, we fuse both features by concatenation before classification. The experimental results show that our multi-modal learning method achieve 4.3381% EER, 0.27 FMR1000, and 0.33 ZeroFMR on IIIT-D Kinect RGB-D Face dataset for face verification and 99.7% classification accuracy, which exceeds the most state-of-the-art methods. Moreover, the global descriptors of model output are designed to be binarized. Our method requires less memory and computation time.