Performance of seven ECG interpretation programs in identifying arrhythmia and acute cardiovascular syndrome

BACKGROUND: No direct comparison of current electrocardiogram (ECG) interpretation programs exists. OBJECTIVE: Assess the accuracy of ECG interpretation programs in detecting abnormal rhythms and flagging for priority review records with alterations secondary to acute coronary syndrome (ACS). METHODS: More than 2,000 digital ECGs from hospitals and databases in Europe, USA, and Australia, were obtained from consecutive adult and pediatric patients and converted to 10 s analog samples that were replayed on seven electrocardiographs and classified by the manufacturers' interpretation programs. We assessed ability to distinguish sinus rhythm from non-sinus rhythm, identify atrial fibrillation/flutter and other abnormal rhythms, and accuracy in flagging results for priority review. If all seven programs' interpretation statements did not agree, cases were reviewed by experienced cardiologists. RESULTS: All programs could distinguish well between sinus and non-sinus rhythms and could identify atrial fibrillation/flutter or other abnormal rhythms. However, false-positive rates varied from 2.1% to 5.5% for non-sinus rhythm, from 0.7% to 4.4% for atrial fibrillation/flutter, and from 1.5% to 3.0% for other abnormal rhythms. False-negative rates varied from 12.0% to 7.5%, 9.9% to 2.7%, and 55.9% to 30.5%, respectively. Flagging of ACS varied by a factor of 2.5 between programs. Physicians flagged more ECGs for prompt review, but also showed variance of around a factor of 2. False-negative values differed between programs by a factor of 2 but was high for all (>50%). Agreement between programs and majority reviewer decisions was 46-62%. CONCLUSIONS: Automatic interpretations of rhythms and ACS differ between programs. Healthcare institutions should not rely on ECG software "critical result" flags alone to decide the ACS workflow.