What question did this study set out to answer?

This research aims to develop a universal representation model for joint image and text understanding through pre-training.

September 25, 2019Open Access

UNITER: UNiversal Image-TExt Representation Learning

View Full Paper

Ask AI

Bookmark

This research aims to develop a universal representation model for joint image and text understanding through pre-training.
Large-scale pre-training over four image-text datasets: COCO, Visual Genome, Conceptual Captions, SBU Captions.
Implemented four pre-training tasks: Masked Language Modeling (MLM), Masked Region Modeling (MRM), Image-Text Matching (ITM), and Word-Region Alignment (WRA).
Used conditional masking and Optimal Transport (OT) to improve alignment during pre-training.
Achieved state-of-the-art performance across six Vision-and-Language tasks including Visual Question Answering and Image-Text Retrieval.
Demonstrated improved performance due to conditional masking and OT-based WRA during pre-training.
Conducted an ablation study to identify the optimal combination of pre-training tasks.

No takes yet. Share an insight, caveat, or question.

Randomized trial shows new state-of-the-art performance in joint image-text tasks, indicating significant advances in multimodal learning.

Chen et al. (2019) studied this question.

View Full Paper

Ask AI

Bookmark

Randomized trial shows new state-of-the-art performance in joint image-text tasks, indicating significant advances in multimodal learning.