Face tampering detection method based on spatiotemporal attention residual network

For the detection of deep forged video, the traditional residual network detection method cannot capture the long-range dependency between video frames and ignores the local critical information. Therefore, we propose a face tampering detection method using a residual network combined with a spatiotemporal attention mechanism. First, we extract video frames using OpenCV, locate facial landmarks on each extracted frame using the Dlib tool, and obtain face frame sequences by cropping, aligning, and resizing faces based on the obtained facial landmarks. Then the spatial domain features of the face data are extracted by removing the residual network (ResNeXt) of the last two layers (global average pooling layer and fully connected layer), based on which the local critical information in the above features is learned by fusing the self-attention mechanism. After that, the long and short-term memory layers capture the long-distance dependencies between video frames to obtain the time-domain features. Finally, some neurons are randomly discarded after the Dropout layer to increase the model's generalization, and a fully-connected layer is used to classify faces as true or false. Experiments are conducted on the FaceForensics++ dataset. The detection accuracy of the method is improved over several baseline algorithms, indicating that the method can effectively detect whether the face region in the video is tampered.