Answers are grounded in this paper and cite their sources.
Automated pneumonia detection from chest radiographs is clinically important, but image-only systems can be brittle when radiographic findings are subtle, labels are report-derived, or contextual clinical evidence is unavailable at deployment. We propose a Modality-Dropout Cross-Attention Transformer (MDCAT), a multimodal architecture that combines chest X-ray image tokens with temporally eligible clinical-note or ED-text tokens using bidirectional crossattention and trains under explicit missing-modality simulation. The model includes modality-availability embeddings, learned absent-modality tokens, auxiliary unimodal heads, and a consistency term that aligns masked-modality predictions with full-input predictions when both modalities are observed. Because pneumonia labels in common chest X-ray datasets are often derived from radiology reports, we specify a leakage-controlled data protocol that separates preimaging clinical text from same-study radiology reports and treats report text only as a secondary ablation. The evaluation design covers full multimodal inference, image-only inference, text-only inference, stochastic modality dropout, real missingness, image corruption, note truncation, calibration, and external image-only validation
No takes yet. Share an insight, caveat, or question.
Jindam et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: