An enhanced multimodal emotion recognition framework using ShuffleNet V2 and efficient channel attention

This paper presents a multimodal emotion recognition framework that integrates enhanced ShuffleNet V2 with an efficient channel attention (ECA) mechanism to mitigate modal redundancy and semantic misalignment. For video modality, an improved ShuffleNet V2 extracts spatial features, while for audio modality, a CNN-BiLSTM network coupled with ECA modules captures temporal dependencies and reduces feature redundancy. A simple feature concatenation followed by fully connected layers is employed for cross-modal semantic alignment. Evaluated on the CREMA-D dataset, the proposed framework achieves a mean classification accuracy of 77.73% ± 1.19%, with particularly high recognition rates for anger (86.1%) and happiness (83.3%). Cross-dataset validation on IEMOCAP Session 1 yields an accuracy of 73.26%, confirming generalization capability. Moreover, the model attains 84 frames per second with only 2.74 million parameters, demonstrating superior computational efficiency and robustness. These results indicate the framework’s practical potential for real-time human–computer interaction and mental health monitoring.