Multimodal fusion framework for enhanced diagnosis of heart failure using ECG, chest X-rays, blood biomarkers, and clinical text data

Heart failure (HF) remains a leading cause of morbidity and mortality worldwide, and conventional diagnostic pathways often underutilize complementary evidence across modalities. This study presents a multimodal fusion framework that integrates electrocardiograms (ECG), chest X-rays (CXR), blood biomarkers, and clinical narratives to enhance HF diagnosis. ECG signals are modeled using a one-dimensional convolutional neural network (1D-CNN), CXR images are processed with a ResNet-based backbone, biomarker panels are learned with XGBoost, and clinical text is represented using BioBERT embeddings to capture clinical context. A two-stage fusion scheme comprising intermediate (feature-level) fusion and late (decision-level) fusion combines modality-specific evidence. Performance was quantified using standard diagnostic metrics under matched experimental protocols. Relative to the strongest unimodal baselines, the intermediate-fusion configuration achieved an accuracy of 0.957, a sensitivity of 0.962, a specificity of 0.948, an F1-score of 0.957, and an AUROC of 0.975, and it consistently outperformed late fusion. A comparative analysis against representative baselines using similar datasets and metrics indicated consistent gains in discriminative performance. SHapley Additive exPlanations (SHAP) analysis showed that ECG dynamics and key biomarkers contributed most strongly to the predictive decisions, with CXR and text providing complementary signals. These results suggest that fusing heterogeneous clinical data can streamline HF diagnosis and support more personalized management.