An ECG foundational model demonstrated preserved diagnostic accuracy for left ventricular systolic dysfunction (AUROC 0.85-0.90) across diverse demographic groups, even when trained on biased datasets.
Does an ECG Foundational Model pretrained with self-supervised learning maintain diagnostic accuracy for LVSD when trained on demographically imbalanced datasets?
A transformer-based ECG foundational model pretrained with self-supervised learning maintains robust diagnostic accuracy for LVSD across diverse demographic groups, mitigating bias from imbalanced training data.
Effect estimate: AUROC 0.85-0.90
Abstract Aims Artificial intelligence (AI)-enabled electrocardiograms (AI-ECG) can detect left ventricular systolic dysfunction (LVSD), but demographic imbalance in training datasets may introduce bias. Foundational models, pretrained on large and diverse datasets, may mitigate such concerns. We aimed to assess the impact of demographic composition in training datasets on the performance of an ECG Foundational Model (ECGFM) for diagnosing LVSD. Methods and results We developed an ECG foundational model (ECGFM) using transformer architecture and self-supervised pretraining on 983,200 ECGs. Using 44,815 paired ECG-echocardiogram datasets, we trained the model under three biased scenarios: (1) sex-skewed (male-only or female-only), (2) race-skewed (White-only or non-White), and (3) balanced. Models were evaluated on a test cohort consisting of 4,663 male patients (52%) and 4,300 female patients (48%) for the sex configuration and 4,440 (49.5%) White, 558 (6.2%) Black, 925 Asian (10.3%), and 3,040 other (33.9%) patients, for the race-based configuration using area under the receiver operating characteristic curve (AUROC). The ECGFM demonstrated consistent performance across all demographic configurations. Training on male-only or female-only cohorts yielded comparable AUROC scores of 0.85–0.90 for both sexes in the test set in predicting LVSD. Similarly, training on white-only or non-white cohorts resulted in robust AUROC scores (≥0.90) across all racial groups, including Asian, Black, Hispanic/Latino, and American Indian/Native Alaskan subgroups. Balanced and imbalanced training produced comparable accuracy, sensitivity, and specificity. The performance of the model was externally tested in EchoNext revealing AUROC scores 0.823-0.917 for sex and 0.822-0.917 for race. Conclusion Our transformer-based ECG foundational model pretrained using self-supervised learning demonstrated preserved diagnostic accuracy for LVSD across diverse demographic groups, even when trained on demographically imbalanced datasets.
Hsieh et al. (Wed,) conducted a other in Left ventricular systolic dysfunction (n=44,815). ECG Foundational Model (ECGFM) vs. Demographically balanced training datasets was evaluated on Diagnosing left ventricular systolic dysfunction (AUROC 0.85-0.90). An ECG foundational model demonstrated preserved diagnostic accuracy for left ventricular systolic dysfunction (AUROC 0.85-0.90) across diverse demographic groups, even when trained on biased datasets.