A Multi-Prior Fusion Network for Video-based Micro-Expression Recognition

Abstract

The analysis of facial micro-expressions (MEs) has emerged as a significant application and topic within the field of image and video processing. However, challenges persist due to the brief duration and subtle intensity of these spontaneous expressions. This paper presents a novel multi-prior fusion network (MPFNet) for ME recognition based on a progressive training strategy. During the prior learning phase, our model is trained using a dual-stream architecture to capture both generic and advanced ME features. In the classification phase, we merge two pre-trained models with complementary prior knowledge and employ weighted fusion for classification within a meta-learning framework. Additionally, this study employs inflated 3D ConvNets (I3D) as a feature encoder and integrates Coordinate Attention (CA) blocks to enhance the automatic learning of spatiotemporal and channel features of ME video sequences. Extensive experiments conducted on three benchmark datasets validate the effectiveness of our model.