Enhanced FusionNet: a deep learning structure for multimodal input-based emotion-aware deepfake speech identification

Voice-based deepfakes pose a significant security concern for voice-based systems and require effective methods for distinguishing genuine and synthetic speech. This study presents Enhanced FusionNet, an emotion-aware deep learning framework for deepfake speech detection that combines Log-Mel-based acoustic representations with MultiScaleCNN, Bidirectional Long Short-Term Memory (BiLSTM), Multi-Head Cross-Attention, Residual Feature Fusion, and Adaptive Gated Fusion. An emotion representation is learned separately from the RAVDESS dataset using a speaker-independent emotion encoder and subsequently transferred to the FoR-based deepfake detection framework. On the independent FoR hold-out test set, Enhanced FusionNet achieved 98.67% accuracy, 98.18% precision, 99.17% recall, 98.67% F1-score, and 99.88% ROC-AUC. Five-fold cross-validation produced a mean accuracy of 99.08 ± 0.35% and ROC-AUC of 99.94 ± 0.05%. The model further achieved 99.40% sensitivity, 98.75% specificity, and 0.91% EER in the out-of-fold evaluation. Component-level ablation analysis demonstrated the varying contributions of the major architectural components, while a controlled FoR-only versus FoR + Emotion experiment showed that the transferred emotion representation provided complementary information but did not improve overall detection performance under the evaluated protocol. The findings demonstrate the effectiveness of Enhanced FusionNet under the evaluated benchmark conditions, while also highlighting the need for external cross-dataset and unseen-generation evaluation. Future work will focus on multilingual and cross-domain evaluation, contemporary anti-spoofing baselines, unseen synthesis methods, and computationally efficient variants for real-time deployment.