CARDIAC-FM: A Multimodal Foundation Model for Cardiovascular Risk Prediction Using ECG and Cardiac MRI

Figures

• 

  Figure 1:. Overview of the CARDIAC-FM framework. A, Multimodal pretraining. ECG waveforms are processed through the ECG Foundation Model (ECG-FM) encoder and MRI (short-axis (SAX) stack, four-chamber (4CH), and two-chamber (2CH) views; 25 frames each) are processed through a convolutional neural network–long short-term memory (CNN-LSTM) encoder. The resulting representations are aligned in a shared latent space using contrastive loss. B, Downstream prediction. The pretrained ECG encoder can be used independently with ECG input alone (left) or combined with the pre-trained MRI encoder for multimodal prediction (right). Both configurations can further incorporate established clinical risk scores, enabling flexible deployment across clinical settings with varying data availability.

• 

  Figure 2:. Prediction of incident atrial fibrillation and heart failure in the UK Biobank. Discrimination (AUROC, left) and precision–recall (AUPRC, right) for 5-year prediction of incident atrial fibrillation and heart failure in the held-out test set. Models compared include clinical risk scores alone (CHARGE-AF for atrial fibrillation; PREVENT-HF for heart failure), an ECG foundation model without multimodal pretraining (ECG-FM), and CARDIAC-FM under four input configurations: ECG only, ECG + MRI, ECG + clinical risk score, and ECG + MRI + clinical risk score. Error bars indicate 95% confidence intervals. CARDIAC-FM combined with clinical risk scores outperformed unimodal baselines across most comparisons.

• 

  Figure 3:. External validation of CARDIAC-FM in the (a) Cardiovascular Health Study and (b) Multi-Ethnic Study of Atherosclerosis. Zero-shot and few-shot prediction performance for 5-year incident atrial fibrillation and heart failure, evaluated using AUROC and AUPRC. Models were pretrained on UK Biobank and externally validated on CHS and MESA. Zero-shot denotes direct application without cohort-specific training; few-shot denotes fine-tuning on 20% of the external cohort data, respectively. Error bars represent 95% confidence intervals. CARDIAC-FM (ECG + Risk Score) achieved the strongest performance across all settings.

• 

  Figure 4:. Risk stratification performance of CARDIAC-FM in external validation cohorts. Hazard ratios for 5-year incident atrial fibrillation, heart failure, heart failure with preserved ejection fraction (HFpEF), and heart failure with reduced ejection fraction (HFrEF) in the Cardiovascular Health Study (CHS; a) and 5-year incident atrial fibrillation and heart failure in the Multi-Ethnic Study of Atherosclerosis (MESA; b). Participants were stratified into risk tertiles based on clinical risk scores alone (CHARGE-AF for atrial fibrillation; PREVENT-HF for heart failure) or CARDIAC-FM (ECG + Risk Score) under zero-shot and 20% fine-tuning conditions. Hazard ratios were estimated using univariable Cox proportional hazards models with the low-risk tertile as reference. Teal points indicate high-risk versus low-risk tertile; red points indicate intermediate-risk versus low-risk tertile. Error bars indicate 95% confidence intervals. In CHS, CARDIAC-FM outperformed clinical risk scores in zero-shot evaluation, with further improvement upon fine-tuning. In MESA, lower event rates, particularly for heart failure, resulted in wider confidence intervals; CARDIAC-FM achieved similar performance to…

• 

  Figure 5:. Evaluation of pre-trained representations across a broad range of cardiovascular outcomes. AUROC (left) and AUPRC (right) for CARDIAC-FM (blue) and ECG-FM (red) across five cardiovascular outcomes at 5- and 10-year prediction horizons in CHS (a) and MESA (b). Outcomes include myocardial infarction (MI), ischaemic stroke (IS), cardioembolic stroke (CES; 10-year only due to limited events at shorter follow-up), cardiovascular death (CV death), and all-cause death (death). Both models used ECG-only input and were fine-tuned using 20% of data from each cohort. CARDIAC-FM consistently outperformed ECG-FM across all outcomes and time horizons in both cohorts. Error bars indicate 95% confidence intervals.