HeartFusion: A Cross-Modality Transformer for Accurate Cardiovascular Disease Diagnosis and Severity Assessment

Cardiovascular disease (CVD) requires diagnostic models that can integrate complementary electrophysiological and anatomical information while also representing ordered disease severity. HeartFusion was developed as a mathematically informed cross-modality Transformer framework combining electrocardiogram (ECG) and cardiac magnetic resonance imaging (MRI). Wavelet-derived ECG time-frequency representations were encoded with Cardioformer, MRI structure was encoded with TransUNet, and modality-specific embeddings were aligned in a shared latent space using cosine-similarity-based cross-modal attention. A transformer classifier performed diagnostic prediction, while an ordinal regression head modeled four ordered severity stages. Three controlled pseudo-paired benchmark cohorts were constructed from PTB-XL with ACDC (CVD-I), MIMIC-IV-ECG with CMRxRecon (CVD-II), and PTB-XL/MIMIC-IV-ECG with OCMR/Cardiac Atlas Project data (CVD-III). Source observations were partitioned before pseudo-pair construction to prevent cross-partition leakage, and five matched random seeds were used for evaluation. HeartFusion achieved diagnostic accuracies of 97.73%, 98.23%, and 98.02%, with AUC values of 98.12%, 98.76%, and 98.39% on CVD-I, CVD-II, and CVD-III, respectively. Severity staging achieved ordinal accuracies above 90%, with the best Cohen's kappa of 0.839 and MAE of 0.340. These findings support phenotype-level multimodal representation learning, while validation on genuinely patient-matched ECG-MRI cohorts remains necessary before clinical deployment.