Key result
Multimodal late fusion models outperform unimodal baselines for 12-month mortality prediction in PE.
Why the study?
Most existing prognostic models for pulmonary embolism rely on a single data modality, and the optimal fusion strategy for multimodal approaches remains unclear.
Do multimodal fusion models improve mortality prediction compared to unimodal models in patients with pulmonary embolism?
Do multimodal fusion models improve mortality prediction compared to unimodal models in patients with pulmonary embolism?
Multimodal AI models, particularly those using late fusion strategies, significantly improve mortality prediction in pulmonary embolism compared to single-modality models.
Answers are grounded in this paper and cite their sources.
Supports multimodal fusion for PE prognosis; hypothesis-generating and requires prospective validation before practice change.
Pulmonary embolism (PE) is a life-threatening condition for which accurate risk stratification is essential for clinical decision-making. Most existing prognostic models rely on a single data modality, typically structured electronic health records (EHR), limiting their ability to capture the heterogeneous nature of the disease. Multimodal approaches combining EHR, computed tomography (CT) imaging, and free-text clinical reports provide complementary information, but the optimal fusion strategy for prognosis prediction remains unclear. In this study, we present a comprehensive comparative study of multimodal fusion strategies for PE prognosis using the INSPECT dataset, which includes structured EHR, CT images, and clinical text with mortality labels. We develop unimodal baselines and multiple bimodal and trimodal models using early, intermediate, late, and cross-fusion strategies within a unified framework. Models are evaluated for 1-, 6-, and 12-month mortality prediction. Multimodal models consistently outperform unimodal baselines across all prediction horizons, with MCC improvements up to +0.13 for 1-month mortality (0.399 vs 0.269) and +0.11 for 6-month mortality (0.479 vs 0.367). The best overall performance is achieved by late fusion models, reaching an MCC of 0.497 for 12-month prediction, compared to 0.454 for the best unimodal model. Interestingly, bimodal combinations often outperform trimodal fusion in short-term prediction, suggesting that improper integration of multiple modalities may introduce noise and reduce effectiveness. These results highlight the importance of fusion design in multimodal clinical AI and provide practical guidance for developing robust prognosis models for pulmonary embolism. Code is publicly available at https://github.com/arco-group/INSPECT-CS.
No takes yet. Share an insight, caveat, or question.
Paolo et al. (2026) studied Pulmonary embolism. Multimodal fusion models (EHR, CT imaging, and clinical text) vs. Unimodal baselines was evaluated on 1-, 6-, and 12-month mortality prediction. Multimodal late fusion models outperformed unimodal baselines for 12-month mortality prediction in pulmonary embolism (MCC 0.497 vs 0.454).
Synapse has enriched 10 related papers. Consider them for comparative context: