Answers are grounded in this paper and cite their sources.
Emotion dynamics formulates principles explaining the emotional fluctuation during conversations. Recent studies explore the emotion dynamics from the self and inter-personal dependencies, however, ignoring the temporal and spatial dependencies in the situation of multi-modal conversations. To address the issue, we extend the concept of emotion dynamics to multi-modal settings and propose a Dialogue Transformer for simultaneously modeling the intra-modal and inter-modal emotion dynamics. Specifically, the intra-modal emotion dynamics is to not only capture the temporal dependency but also satisfy the context preference in every single modality. The intermodal emotional dynamics aims at handling multi-grained spatial dependency across all modalities. Our models outperform the stateof-the-art with a margin of 4%-16% for most of the metrics on three benchmark datasets.
No takes yet. Share an insight, caveat, or question.
Mao et al. (2021) studied this question.