This is the authors' abstract. We don't add key points for this paper.
Multimodal data plays a crucial role in our daily lives. It comprises heterogeneous data, such as audio, visual, language, etc., that surrounds us in the world. Machine learning for single modality data can result in limitations to performance owing to a lack of information, while many applications are more suitable to be modeled as multimodal learning tasks. Audiovisual learning and vision-language modeling have been extensively studied, and recent research is starting to focus on audio-visuallanguage joint modeling. Currently, there is no review focused on audiovisual language joint modeling, even though audio, visual, and language modalities frequently co-occur. Therefore, we survey audio, visual, and language trimodal learning to understand the ability of multimodal learners. Problem formulations and benchmark datasets are introduced. We summarize the state-of-the-art methods for each task with corresponding evaluation criteria, and present current trends of multitask learning, larger model size, as well as pretrain-finetune training paradigm.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: