Automated triage of 12-lead electrocardiograms using deep neural networks: external validation in a Spanish population

Abstract Background Accurate interpretation of the electrocardiogram (ECG) is essential for identifying a wide range of cardiac abnormalities. However, expert analysis is not always available, particularly in primary care and resource-limited settings. Deep learning algorithms offer a potential solution by automating ECG interpretation, but external validation is necessary to assess their generalizability across different patient populations and ECG acquisition devices. Purpose This study aims to externally validate a previously developed deep learning algorithm for triaging 12-lead ECGs (GALVO 12L). The algorithm classifies ECGs into normal, abnormal, subacute, and acute categories while also detecting 37 specific ECG abnormalities and assessing ECG interpretability. Methods An ensemble of deep neural networks was trained using approximately 400,000 12-lead ECGs from the Netherlands, each labeled by a cardiologist. External validation was conducted on ECGs obtained from patients in a general practice setting in Spain, recorded using a different ECG device. All validation ECGs were manually annotated by an independent electrophysiologist. The algorithm’s performance was assessed using the c-statistic (area under the receiver operating curve), sensitivity, specificity, positive predictive value, and negative predictive value. Results A total of 904 patients were included, with 849 ECGs deemed interpretable by the algorithm. The deep learning model demonstrated strong discriminative performance, achieving an overall c-statistic of 0.94 for the triage categories. Performance for individual categories varied, with c-statistics ranging from 0.85 for detecting non-interpretable ECGs to 0.99 for identifying acute ECG abnormalities. Sensitivity for detecting acute ECG abnormalities was 100%, showing that no acute cases were missed. Undertriage and overtriage was rare, occurring in 1.5% and 3.5% of cases, respectively. The cardiologist fully agreed with the algorithm’s detected specific ECG abnormalities in 760 patients (89.5%), while partial agreement was observed in 80 patients (9.5%), and no agreement in only 9 cases (1%). Conclusions This external validation study demonstrates that a previously developed deep learning algorithm for ECG triage exhibits excellent generalizability across different patient populations and ECG acquisition devices. Notably, the model detected all acute ECG abnormalities, ensuring timely identification of critical cases. The low rates of undertriage and overtriage suggest that the algorithm can reliably prioritize high-risk patients while minimizing unnecessary referrals. Implementing this algorithm in clinical practice could enhance time-to-treatment for acute cardiac conditions, optimize resource allocation, and reduce the burden on healthcare professionals.Confusion matrix Overview of performance metrics