Multimodal aspect-based sentiment analysis network with adaptive modality balancing

Multimodal aspect-based sentiment analysis, as a fine-grained sentiment analysis task, aims to identify the sentiment polarity associated with a given aspect entity. Current methods mainly focus on fusing information from different sources (e.g., image and text). However, in real-world scenarios, the emotional contributions of different modalities are not balanced. Therefore, directly exploring modality fusion inevitably has its limitations. To address this issue, we propose a multimodal aspect-based sentiment analysis network with adaptive modality balancing (AMB), which adaptively analyses cross-modal emotional contributions. Specifically, we first separately aggregate the feature information of a single modality through self-attention. Furthermore, during modality fusion, each modality is assigned a different weight based on its feature entropy, which helps balance their emotional contributions. Experiments on two public datasets validate the effectiveness of the proposed network.