A Novel Multimodal Sentiment Analysis Model Based on Gated Fusion and Multi-Task Learning

Sentiment analysis is an important research area in Natural Language Processing (NLP). With the explosion of multimodal data, Multimodal Sentiment Analysis (MSA) attracts more and more attention in recent years. How to Effectively harnessing the interplay between diverse modalities is paramount to achieving comprehensive fusion of MSA. However, current research predominantly emphasizes modality interaction, while overlooking unimodal information, thus neglecting the inherent disparities between modalities. To address these issues, we propose a novel model for multimodal sentiment analysis based on gated fusion and multi-task learning. The model adopts multi-task learning to concurrently address both multimodal and unimodal sentiment analysis tasks. Specifically, for the multimodal task, we leverage cross-modal Transformers with gating mechanisms to facilitate modality fusion. Subsequently, the fused representations are harnessed to generate sentiment labels for the unimodal tasks. Experiments on the CMU-MOSI and CMU-MOSEI datasets demonstrate that our model outperforms the existing methods and achieves the state-of-the art performance.