Answers are grounded in this paper and cite their sources.
Emotion and intent joint understanding in multimodal conversations (MC-EIU) aims to model the semantic dependencies among multimodal conversations while inferring emotion and intent information. Despite making progress, existing methods overlook the differential contributions of modalities and rely on single-round interactions for emotion and intent recognition, resulting in suboptimal model understanding performance. To overcome these limitations, we propose MEIPro, a novel progressive interaction and adaptive weight fusion based multimodal joint understanding of emotion and intent framework. We first design a hierarchical denoising module to effectively remove noise and redundant information from multimodal data. Then, we propose an adaptive weight fusion mechanism that dynamically fuses multimodal features by taking the true classification probabilities of each modality as their respective contributions, thus enhancing the fusion process. Additionally, we present a progressive dual task interaction module to capture the deep seated interactions between emotion and intent through a step-by-step multi-round iteration. Experiments on the benchmark MC-EIU bilingual dataset demonstrate that our MEI-Pro framework significantly outperforms state-of-the-art baselines in both emotion and intent tasks. Specifically, on the English dataset, the F1-scores of the multimodal emotion and intent understanding tasks have increased by 6.12% and 7.25% respectively.
No takes yet. Share an insight, caveat, or question.
Li et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: