Attention-Based Fusion of Intra- and Intermodal Dynamics in Multimodal Sentiment Analysis

Sentiment analysis, the process of predicting sentiments expressed in human communication, has evolved to multimodal sentiment analysis (MSA). Recent advances in attention-based MSA models have demonstrated the effectiveness of capturing intramodality and intermodality dynamics. However, challenges remain in achieving optimal performance and creating effective context representations across modalities. To address these challenges, we propose the Multi-Head self-attention with Context-Aware attention model, which utilizes two attention-based mechanisms to strategically capture intramodality dynamics within each modality before delving into intermodality dynamics. The experimental results on 5 datasets show the superiority of our model in comparison with the state of the arts.