TFC-SpeechFormer: Efficient Emotion Recognition Based on Deep Speech Analysis and Hierarchical Progressive Structures

Speech emotion recognition aims to automatically identify emotions in human speech, enabling machines to understand and engage emotional communication. In recent years, Transformers have demonstrated strong adaptability and significant effectiveness in speech recognition. However, Transformer models are proficient at capturing local features, struggle to extract fine-grained details, and often lead to computational redundancy, increasing time complexity. This paper proposes a Speech Emotion Recognition model named Temporal Fusion Convolution SpeechFormer (TFCS).The model comprises a Hybrid Convolutional Extractor (HCE) and multiple Temporal Fusion Convolution SpeechFormer Blocks (TFCSBs). HCE, consisting of an encoder and convolutional modules, enhances speech signals to capture local features and texture information, extracting frame-level features. TFCSBs utilize a newly proposed Temporal Fusion Convolution Module and Speech Multi-Head Attention Module to capture correlations between adjacent elements in speech sequences. TFCSBs utilize feature information captured by HCE to sequentially form frame, phoneme, word, and sentence structures, and integrates them to establish global and local relationships, enhancing local data capture and computational efficiency. Performance evaluation on the IEMOCAP and DAIC-WOZ datasets demonstrates the effectiveness of HCE in extracting fine-grained local speech features, with TFCS outperforming Transformer and other advanced models overall.