Identifying the Emotional State of Tourists in Rural Tourism Videos Using Multimodal Multi-Scale Attention Mechanisms

This study addresses the challenge of accurately identifying tourists’ emotional states from unstructured rural tourism videos under diverse environmental stimuli by incorporating multimodal analysis of visual, acoustic, and textual information. To achieve robust and fine-grained emotion recognition, a Multimodal Multi-Scale Attention Transformer (MMSAT) framework is proposed for collaborative modeling of audiovisual and verbal signals. The framework employs an intra-modal attention mechanism to adaptively fuse multi-scale visual representations while utilizing a three-layer Transformer encoder to capture deep cross-modal interactions and complementary semantic information. The fused representations are subsequently used to predict emotional valence and arousal with high precision. Experimental results on a self-constructed rural tourism video dataset demonstrate that the proposed MMSAT model achieves Concordance Correlation Coefficient (CCC) scores of 0.823 and 0.805 for valence and arousal prediction, respectively, outperforming the strongest baseline model, Multimodal Transformer (MulT), by 0.042 and 0.053. The proposed framework provides an effective solution for objective emotion assessment in complex open environments and offers methodological insights into multimodal information propagation and intelligent signal processing, with potential implications for advanced electromagnetic information perception and cross-domain sensing applications.