What question did this study set out to answer?

To develop robust joint multimodal representations for sentiment analysis that remain accurate even when input modalities are noisy or missing during inference.

July 17, 2019Open Access

Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities

View Full Paper

Ask AI

Bookmark

Authors

HPHai PhamSamsung (United Kingdom)PLPaul Pu LiangMoscow Institute of Thermal Technology TMThomas ManziniTexas A&M University

Discussion

Member takes

Implication

Machine learning study demonstrates enhanced sentiment prediction robustness using cyclic modality translation, indicating resilience to missing sensory inputs at test time.

Key Points

To develop robust joint multimodal representations for sentiment analysis that remain accurate even when input modalities are noisy or missing during inference.
Designed a sequence-to-sequence translation architecture that translates between language, visual, and acoustic modalities.
Incorporated a cycle consistency loss and a coupled translation-prediction objective to preserve maximal cross-modal information during training.
Evaluated model robustness and accuracy on three benchmark datasets: CMU-MOSI, ICT-MMMO, and YouTube.
Established new state-of-the-art performance on the CMU-MOSI, ICT-MMMO, and YouTube multimodal sentiment analysis benchmarks.
Demonstrated that the model learns increasingly discriminative representations with additional modalities while maintaining high accuracy using only a single source modality at test time.

Cite This Study

Pham et al. (2019) studied this question.

synapsesocial.com/papers/6a01cc4760baf37e2cd8bb5d https://doi.org/10.1609/aaai.v33i01.33016892

View Full Paper

Ask AI

Bookmark

Implication

Machine learning study demonstrates enhanced sentiment prediction robustness using cyclic modality translation, indicating resilience to missing sensory inputs at test time.

Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities

Discussion

Member takes

Implication

Key Points

Cite This Study

Found in Translation: Learning Robust Joint Representations by Cyclic Translations between Modalities

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider