Detecting structural heart disease from electrocardiograms using AI

Figures

• 

  Fig. 1. Model development: NYP multicentre cohort derivation. The deep learning model was trained and tested using data from an eight-hospital system (NYP Hospital). ECG data were accessed using the MUSE system with removal of ECGs with missing age, sex and patient identifier, poor study quality designation by machine recommending repeating of ECG or presence of ventricular pacing. Echocardiogram data were accessed using hospital systems with removal of patients with repaired or replaced heart failures. This yielded 1.2 million ECG–echocardiogram pairs in 230,018 unique patients with data split into train, validation and test sets.

• 

  Fig. 2. Multicentre EchoNext performance. Performance of the model in detection of individual and compositive SHDs. a,b, By AUROC (a) and AUPRC (b), the model had high performance in detection of SHD in the internal eight-hospital NYP system test set and three geographically distinct external test sets (Montreal Heart, Cedars-Sinai and University of California San Francisco (UCSF)). c, Individual disease models had the highest performance in the detection of reduced low left ventricular (LV) and right ventricular (RV) systolic function by AUROC with favourable performance for other disease states. d, Assessment of the AUPRC for the individual disease states is highly dependent on the underlying prevalence of the individual disease states. TR, tricuspid regurgitation. Dashed lines (a,c) indicate random classifier.

• 

  Fig. 3. Performance characteristics of EchoNext in retrospective validation, comparison to cardiologists, silent deployment, and clinical trial. a, In test sets of held-out patients at these sites, as well as in three geographically distinct external test sets, the model demonstrated high accuracy, as demonstrated by AUROC. Dashed line indicates random classifier. b, In a survey of ECGs shown to cardiologists to assess for the presence of SHD, the AI model demonstrated superior performance in SHD detection compared with cardiologists alone or cardiologists given the EchoNext risk score (n = 3,200 cardiologist interpretations). Error bars show the CIs derived from results across 13 cardiologists; because the AI model was run once on the entire set of 150 ECGs, there are no error bars for the ‘AI alone’ results. c, This model was evaluated in a temporally distinct held-out set with similar accuracy, with 45% (n = 3,444) of patients labelled as high risk by the model failing to undergo echocardiography as part of routine clinical care. d, The clinical use of AI-ECG to detect SHD was evaluated in a single-arm, single-site, open-label pilot clinical trial, DISCOVERY, with stratified…

• 

  Fig. 4. Characteristics of the released Columbia ECG dataset and performance of the Columbia mini-model trained on these data for SHD prediction. A 100,000 ECG dataset is being released from data from Columbia University Irving Medical Center as part of this paper. These data consist of the ECG waveform, ECG tabular features and paired echocardiographic data. A model trained and tested on this dataset demonstrated similar performance in SHD detection as the multisite model that served as the primary analysis in this study when assessed by AUROC and AUPRC. Dashed line indicates random classifier.