C3D-VIT: Consistency-Aware 3D Vision Transformer for Face Forgery Detection

Abstract

Recently, spatial and temporal inconsistencies have been shown to effectively enhance the generalization performance of face forgery detection, as common forgery strategies create inconsistencies among face regions and across frames. However, current methods often focus on either spatial or temporal modeling and involve complex modules that are difficult to integrate into other frameworks. Some approaches require extra training data or forgery masks to learn consistency features, limiting their applicability and performance. Additionally, most current spatio-temporal methods rely on CNNs, with few designed for transformers, making it challenging to adapt them for transformer use. To address these issues, we propose an efficient consistency modeling block that unifies spatial and temporal consistency modeling within a transformer framework. Specifically, we calculate the reconstruction error between features and predictions via neighboring spatial and temporal dimensions to explicitly model consistency without extra training data or forgery masks. This block can be seamlessly integrated into a transformer framework, adding less than 1% to computing costs.