Multi-Stream Facial Adaptive Network for Expression Recognition from a Single Image

Abstract

Facial expression recognition from a single image has potential applications in fields including human-computer interaction and medical diagnosis. Most recent methods use deep neural networks to directly learn from a roughly cropped facial image which is usually detected from a whole image by face detection algorithms. We observe that unrelated surrounding regions in the rough facial image prevent deep neural networks from learning facial-related discriminate features. To solve this problem, we present a Facial Adaptive Network (FAN) which is able to adaptively select an interest region from the given facial image, thus suffering less from the effect of unrelated regions. Based on the selected interest region, we further apply the self-attention mechanism to learn discriminate facial features. Moreover, we introduce a multi-stream FAN (ms-FAN) that learns richer facial features from multiple interest regions that are selected from pose-augmented facial images. Extensive experiments on Oulu-CASIA, CK+, and RAF-DB datasets consistently verify the effect of our proposed MS-FAN by achieving comparable results with state-of-the-art methods. Our code is available at https://github.com/zhangbc12/DAtt-ViT.