A Comparative Analysis of Deep Learning Models for Multi-class Speech Emotion Detection

Abstract In today's digital age, where communication transcends traditional boundaries, the exploration of deep learning models for Speech Emotion Recognition (SER) holds immense significance. As we increasingly interact through digital platforms, understanding and interpreting emotions becomes crucial. Deep learning models, with their ability to autonomously learn intricate patterns and representations, offer unparalleled potential in enhancing the accuracy and efficiency of SER systems. This project delves into models for multi-class speech emotion recognition on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The RAVDESS dataset contains 1440 speech audio recordings from 24 professional actors, expressing 8 different emotions: neutral, calm, happy, sad, angry, fearful, surprise, and disgust. Models including Deep Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory networks (LSTMs), Gated Recurrent Units (GRUs), Temporal Convolutional Networks (TCNs), and ensembles were developed. Additionally, data augmentation through pitch shifting, noise injection, and a combination thereof expanded the dataset. Besides spectrogram inputs, handcrafted audio features like Mel Frequency Cepstral Coefficients (MFCCs), Chroma Short-time Fourier transform, root mean square, and zero crossing rate were experimented with as inputs to further boost model performance. The best-performing models were a Temporal Convolutional Network (TCN), achieving 96.88% testing accuracy, and a Gated Recurrent Unit (GRU) achieving 97.04% testing accuracy in classifying the 8 emotions, outperforming previous benchmark results on this dataset.