SSTNet: Detecting Manipulated Faces Through Spatial, Steganalysis and Temporal Features

Abstract

Compared to conventional object detection which focuses on high-level image content, face manipulation detection pays more attention to low-level artifacts and temporal discrepancies. However, there are few methods considering both of these two characteristics. In this work, we propose a novel manipulation detection framework, named SSTNet, which detects tampered faces through Spatial, Steganalysis and Temporal features. Spatial features are extracted by a deep neural network for finding visible tampering traces like unnatural color, shape and texture. We propose a constraint on convolutional filters to extract steganalysis features for detecting hidden tampering artifacts like abnormal statistical characteristics of image pixels. Temporal features are extracted by a recurrent network for discovering inconsistency between consecutive frames. Experimental results on Face-Forensics++ dataset demonstrate that SSTNet outperforms other methods and achieves state-of-the-art performance on accuracy and robustness to compression. Furthermore, the generalization capability of SSTNet is verified on the GAN-based DeepFakes dataset.