MSRFormer: Hybrid Scale Self-Attention and Local Fast Convolution Transformer for Facial Expression Recognition

Abstract

With the emergence of the Transformer architecture, the accuracy of deep learning within the domain of facial emotion recognition has seen further enhancement. However, Transformer comes with increased training complexity and time due to the large parameter count. Additionally, the global receptive field in Transformer's attention leads to unnecessary computations for features with limited spatial extent in image sentiment analysis. In this paper we presents a MSRFormer model, which combines Hybrid-scale self-attention and local fast convolution to address existing issues. The Hybrid-scale self-attention enables precise focus on salient regions of the image, where key features are located. A Local Fast Convolution module was integrated into the MLP head of the Transformer model, enhancing training speed and reducing parameters while maintaining feature learning. we conducted experiments on FERPlus, RAF-DB, and KDEF datasets and confirming its strong performance and broad applicability in facial expression recognition.