Optical Flow based CNN for detection of unlearnt deepfake manipulations

• Deepfakes constitute a serious threat in video manipulation. • A sequence-based approach to investigate possible motion dissimilarities in the temporal structure of a video. • Optical Flow fields to exploit inter-frame correlations to be used as input of a CNN classifier. • OF-based detection scheme provides robustness in a cross-forgery scenario. • OF-based method can be integrated with frame-based techniques. A new phenomenon named Deepfakes constitutes a serious threat in video manipulation. AI-based technologies have provided easy-to-use methods to create extremely realistic videos. On the side of multimedia forensics, being able to individuate this kind of fake contents becomes ever more crucial. In this work, a new forensic technique able to detect fake and original video sequences is proposed; it is based on the use of CNNs trained to distinguish possible motion dissimilarities in the temporal structure of a video sequence by exploiting optical flow fields . The results obtained highlight comparable performances with the state-of-the-art methods which, in general, only resort to single video frames. Furthermore, the proposed optical flow based detection scheme also provides a superior robustness in the more realistic cross-forgery operative scenario and can even be combined with frame-based approaches to improve their global effectiveness.