Deepfake Face Recognition Through Modified and Imporved Deep Transfer Learning

The manipulation of photo, audio, and video content has been a topic of interest for many years, as people uses fake faces to indulge in various immoral act like pornography, fraud, and defamation. In the early days, classification of real and fake faces was done using traditional methods such as editing frames by frame or using chroma keying, this traditional approach is time consuming and lacks enough editors that have the technical skills to do the frame-by-frame edition or use the chroma keying. With technological advancement, new techniques have been developed that allow for much more sophisticated and realistic manipulations, one such technique is deepfakes. Deepfakes are created using deep learning algorithms to swap or replace faces in videos or images. This can be done with a high degree of realism, making it difficult to distinguish between real and fake content. This research aims to develop a deep fake detection system using deep transfer learning (modified VGG19 and ResNet50 models), these two models were chosen over other CNN architectures due to their proven better performance, faster recognition time and lesser memory usage. The research modified the original VGG19 and ResNet architectures by replacing the last five layer with a customized dense layers that will help with faster and accurate recognition of faces. A balanced dataset comprising 70,000 real faces from the Flickr dataset and 70,000 fake faces generated by StyleGAN was utilized. This research employed hold-out evaluation method. VGG19 gave an accuracy, f1score of 91.59% and 91.47% respectively while RestNet50 gave an average accuracy and F1score of 96.61% and 96.59% respectively on the testing dataset. This shows that ResNet 50 gave the best performance both on the training, validation and testing dataset. The developed system was also compared with other state-of-art methods and they were all outperformed.