Assessing the external performance of AI-ECG for left ventricular systolic dysfunction: a review and multinational head-to-head comparison

Abstract Background Artificial intelligence (AI) applied to electrocardiograms (ECGs) has shown promise in detecting left ventricular systolic dysfunction (LVSD). Despite the development and validation of numerous AI-ECG models, concerns about generalizability and potential bias persist due to incomplete reporting of dataset characteristics and limited model sharing, which hinder independent external validation. Purpose This study aims to (1) identify all AI-ECG models for LVSD detection through a systematic review and (2) evaluate their performance within a single external validation dataset. Methods A systematic review was conducted to identify AI-ECG models for LVSD detection. Ovid MEDLINE(R) was searched in October 2023 from 2010 onward for studies reporting model derivation or external validation of AI-ECF for LVSD detection. Extracted data included model architecture, cohort characteristics, and performance metrics. All corresponding authors from included articles were contacted and invited to collaborate by sharing their models for external validation. Performance of shared models was evaluated using an independent cohort with paired ECG and cardiac MRI (CMR) data. To ensure robustness, model performance was first assessed in the full CMR cohort, representing a highly complex population. Additionally, to reduce selection bias related to clinical indication for CMR, a subgroup was created to reflect a representative heart failure cohort, ensuring that 15% of patients had a left ventricular ejection fraction below 40%. Performance was assessed using the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV). Results were stratified by age, sex, and ECG characteristics to evaluate potential biases. Results Systematic review identified 35 studies describing 51 AI-ECG models. Most reported strong internal validation performance, with 29 (57%) reporting AUC values 0.9. External validation was reported in 20 studies, with generally stable performance (Figure 1). 1,306 individuals (mean age 59 ±15 years, 35% female) were included as the independent validation cohort. Four research groups from Korea, Taiwan, the United States, and the Netherlands provided their models. AUC values ranged from 0.83 to 0.93 in the full CMR cohort, and from 0.87 to 0.96 in the representative heart failure cohort (Figure 2). Model performance remained stable across sex and age groups but was slightly lower in ECGs with QRS duration 120ms or atrial fibrillation. Conclusion Systematic review demonstrated AI-ECG models report strong performance for LVSD detection in both internal and external validation. The four AI-ECG models we externally validated showed consistent performance. We encourage more researchers to share models for independent external validation.Figure 1 Figure 2