Improving Micro-expression Recognition using Multi-sequence Driven Face Generation

Abstract

Micro-expression (ME) recognition holds great potential for revealing true human emotions. A significant barrier to effective ME recognition is the lack of sufficient annotated ME video data because MEs are subtle and involuntary facial expressions that are very hard to capture. To address this issue, data augmentation techniques, such as ME migration based on a driven video, have been employed to enrich training samples. Considering that MEs can be complex facial movements involving multiple action unit (AU) changes, we propose a novel ME generation approach that enables the creation of more realistic facial sequences by fusing MEs from multiple videos rather than just single driven video. To enhance the effectiveness of multi-sequence ME transfer, we adapt the thin plate spline motion model and improve traditional face alignment methods to better suit the model, facilitating multi-sequence driven ME generation. In our experiments, we conduct a downstream ME recognition task using models trained on our augmented ME sequences to demonstrate the effectiveness of our approach on the SAMM, SMIC, and CASME II datasets. The results confirm that our proposed approach outperforms state-of-the-art (SOTA) augmentation and generation methods in terms of F1 score and recognition accuracy.