A deformable convolution and attention-driven multiscale transformer framework for robust and explainable speech emotion recognition

Speech Emotion Recognition (SER) is a key contribution to Human-Computer interaction (HCI), mental health diagnosis, and affective computing. However, SER remains challenging due to challenges, including non-stationarity of speech signals and speaker variability. This research introduces a deformable convolution-enhanced hierarchical Swin Transformer network (DUST-Net) with token pruning and uncertainty-aware attention for robust SER. This integrates wavelet-based multiscale feature extraction, deformable convolutions, and adaptive token pruning to capture local and global emotional features efficiently. Signal preprocessing, Hiking Optimization Algorithm-based optimization, and DeepLIFT interpretability further improve performance and transparency. Experimental results achieved 98.75% accuracy, demonstrating robustness, generalizability, and efficiency in emotion recognition.