Enhanced multilingual speech emotion recognition using a modified DenseNet-169 with deformable convolution and staged feature fusion for Indo-European languages

Speech emotion recognition (SER) poses significant challenges in a multilingual context. It is evident in the literature that models proposed by researchers for emotion classification from speech often perform well in particular language(s) but struggle to maintain accuracy across languages. In this paper, we propose an improved DenseNet-169-based SER model that enhances performance by integrating deformable convolution and a progressive feature fusion strategy that combines temporal and spectral features, enabling more accurate emotion classification across multiple languages. To evaluate the performance of the proposed model, we have selected six linguistically diverse Indo-European languages. The result shows a remarkable improvement in emotion classification, achieving an accuracy of 99.65% on the RAVDESS database, 98.96% on the SAVEE database, 98.13% on the Emo-DB database, 95.86% on the SUBESCO (Bengali, an Indian and Bangladeshi language) database, 95.78% on the EMOVO database, and 97.50% on the URDU (an Indo-Aryan language spoken in South Asia) database with low error rates (<0.08 across all datasets) as reported in the state of the art. To further evaluate the robustness and generalization of the proposed model, we have tested it on different cross-corpora and found significant performance in classification accuracy.