Robust deepfake video detection using spatio-temporal features and dynamic difference learning

Abstract Recent progress in facial manipulation technologies has made deepfake videos increasingly convincing, posing significant challenges to detection systems that rely solely on analyzing individual frames. Consequently, there has been a growing emphasis on investigating spatial and temporal inconsistencies within video sequences to more accurately distinguish between genuine and manipulated content. However, many existing approaches still depend on combining frame-level and sequence-level features without adequately addressing irregularities in facial motion, which can significantly constrain detection performance. To overcome these limitations, we propose a comprehensive deep learning framework that integrates both spatial and temporal analysis. Facial landmarks are extracted from each video frame using Dlib’s 68-point detector, providing geometric descriptors of facial structure. These landmarks are fed into a Transformer encoder to capture both short- and long-term motion dynamics, enhanced by a Dynamic Difference Module (DDM) that emphasizes abrupt, unnatural changes. Meanwhile, CNNs extract spatial features, and LSTMs model temporal dependencies. The performance was assessed using standard metrics, including precision, recall, and F1-score, ensuring a comprehensive evaluation of the framework’s effectiveness. Beyond a study, the full framework was experimentally validated across three benchmark datasets—FaceForensics++ (FF++), UADFV, and DFDC—achieving a remarkable 100% accuracy, thereby demonstrating its robustness and strong generalization capability.