Answers are grounded in this paper and cite their sources.
Social media posts during disasters often pair a short message with an image, and combining the two is widely assumed to help automatic systems decide which posts carry useful information for responders. We test that assumption on the CrisisMMD benchmark and ask a more specific question: when does fusion actually help, and which fusion method holds up on the cases that matter? Using a frozen CLIP backbone with five trainable heads (two single-modality baselines, late concatenation, cross-modal attention, and an agreement-gated variant), we evaluate informativeness classification over five random seeds with paired significance tests. Our analysis yields three findings. First, fusion improves over the best single-modality model on the full test set, and the gain is significant for attention-based fusion. Second, this gain comes almost entirely from samples where the image and text labels agree, on the roughly one-third of pairs where they disagree, every fusion model drops sharply, and a text-only model is in fact the most accurate. Third, within the disagreement set the fusion method matters, cross-modal attention is significantly more robust than plain concatenation, while an explicit agreement gate adds no significant gain over attention alone. We report the negative result on gating openly since it shows that the robustness comes from attention itself. The study gives a reproducible, statistically grounded account of where multimodal fusion earns its place in crisis informatics.
No takes yet. Share an insight, caveat, or question.
Ghimire et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: