A Unified Perspective on Multimodal Emotion Recognition, Personality Trait Assessment, and Ambivalence/Hesitancy Recognition: Corpora, Methods, Evaluation, and Open Challenges

Multimodal analysis of human behavior has advanced rapidly, yet emotion recognition (ER), personality trait assessment (PTA), and ambivalence/hesitancy recognition (AHR) are usually reviewed separately. This review jointly examines these related but distinct tasks through a comparative analysis of corpora, computational methods, evaluation protocols, and ethical limitations. Structured searches in Scopus and Web of Science were followed by multi-stage screening and multi-reviewer assessment. The final methodological sample comprised 56 unique studies published between 1 January 2023 and 1 August 2026. One study addressed both ER and PTA, resulting in 57 task-specific entries. The reviewed methods include cross-modal interaction, temporal and contextual modeling, graph-based reasoning, reliability-aware fusion, learning under limited supervision, and semantic enrichment with large language models. The reviewed ER studies address contextual modeling, robustness-related objectives, and open-vocabulary prediction. PTA studies focus on structured behavioral modeling, temporal aggregation, target-aware assessment, and generalization-related strategies. AHR remains less explored and is concentrated on a narrow benchmark landscape with distinct frame- and video-level protocols. A post-inclusion quality and reproducibility assessment showed explicit repeated-run uncertainty reporting in only 10 of 56 studies. Explicit cross-corpus or external validation was reported in only 3 of 56 studies. These findings indicate that many methodological trends remain supported primarily by within-benchmark evidence. The synthesis identifies shared multimodal processing principles while preserving differences in target stability, annotation sources, temporal granularity, and evaluation. Key priorities include cross-corpus generalization, uncertainty, interpretability, cultural validity, privacy, and human oversight.