An artificial intelligence–enabled ECG algorithm for comprehensive ECG interpretation: Can it pass the ‘Turing test’?

Q: What is the clinical evidence from this study?

Study design: Observational. Population: Adults requiring standard 12-lead ECG (n=500). Intervention: AI-ECG algorithm vs. Marquette 12SL automated computer program. Primary outcome: Mean interpreter composite score for diagnostic accuracy (1=ideal, 2=acceptable, 3=unacceptable) (p=<0.0001).

Q: What question did this study set out to answer?

To develop an artificial intelligence algorithm capable of comprehensive, human-like 12-lead ECG interpretation and evaluate its performance against standard automated software and clinical interpretations.

Anthony H. Kashou; Siva K. Mulpuru; Abhishek Deshmukh; Wei-Yin Ko; Zachi I. Attia; Rickey E. Carter; Paul A. Friedman; Peter A. Noseworthy

doi:10.1016/j.cvdhj.2021.04.002

May 5, 2021Cardiovascular Digital Health JournalOpen Access

An artificial intelligence–enabled ECG algorithm for comprehensive ECG interpretation: Can it pass the ‘Turing test’?

View Full Paper

Ask AI

Bookmark

Share

Key result

The AI-ECG algorithm improved diagnostic accuracy compared to the Marquette 12SL automated computer program, reducing the mean interpreter composite score from 1.497 to 1.377 (p<0.0001).

Figures

Why the study?

The study was conducted to develop an AI-enabled ECG algorithm capable of comprehensive, human-like ECG interpretation and evaluate its performance against conventional ECG interpretation methods.

Does an AI-enabled ECG algorithm improve comprehensive 12-lead ECG interpretation accuracy compared to a standard automated computer program in adult patients?

Key Points

To develop an artificial intelligence algorithm capable of comprehensive, human-like 12-lead ECG interpretation and evaluate its performance against standard automated software and clinical interpretations.
Trained an AI algorithm on nearly 2.5 million standard 12-lead ECGs from over 720,000 adult patients at the Mayo Clinic ECG laboratory between 2007 and 2017.
Assessed performance across 500 randomly selected ECGs from 500 patients, comparing the Marquette 12SL automated computer program, the AI-ECG algorithm, and final clinical interpretations.
Had 3 blinded cardiac electrophysiologists adjudicate each interpretation as ideal (no changes needed), acceptable (minor edits needed), or unacceptable (major edits needed).
Major edits (unacceptable) were required in an average of 202 (13.5%) computer program interpretations, 123 (8.2%) AI-ECG interpretations, and 90 (6.0%) final clinical interpretations.
Cardiologists adjudicated 958 (63.9%) computer-generated, 1058 (70.5%) AI-ECG, and 1118 (74.5%) clinical interpretations as ideal.
Mean composite scores were 1.497 for computer, 1.377 for AI-ECG, and 1.315 for clinical interpretations, with AI-ECG outperforming the computer program (Bonferroni-adjusted P=.0001).

Study Design

Type

Observational (n=500)

Blinding

Blinded (for adjudication by cardiac electrophysiologists)

Multicenter

No

Structured PICO

Does an AI-enabled ECG algorithm improve comprehensive 12-lead ECG interpretation accuracy compared to a standard automated computer program in adult patients?

P

Population

500 randomly selected adults with standard 12-lead ECGs used to compare the diagnostic accuracy of an AI-ECG algorithm against a standard automated program.

I

Intervention

Novel AI-enabled ECG (AI-ECG) algorithm capable of complete 12-lead ECG interpretation

C

Comparator

Marquette 12SL automated computer program and final clinical interpretations

O

Outcome

Diagnostic accuracy of each ECG interpretation method based on accuracy scores (1=ideal, 2=acceptable, 3=unacceptable) provided by 3 expert over-reading cardiologists

Main Result

Absolute Event Rate: 1.377% vs 1.497%

p-value: p=<0.0001

An AI-ECG algorithm provides more accurate comprehensive 12-lead ECG interpretations than a standard automated computer program, more closely approximating expert cardiologist over-read.

Limitations

Derived on randomly selected patients from a single institution, which may not reflect all ethnic and racial groups
Used error-prone over-reading cardiologist interpretation as the 'gold standard'
Limited number of participating expert interpreters (3)
Compared against a single computerized ECG interpretation system
Single center derivation (Mayo Clinic), may not reflect all ethnic and racial groups
Gold standard was error-prone over-reading cardiologist interpretation
Small number of expert interpreters (3) volunteered
Single computerized ECG interpretation system used as comparator

Ask about this paper

Answers are grounded in this paper and cite their sources.

Implication

May reduce major ECG interpretation edits; hypothesis-generating and requires prospective validation before practice change.

Abstract

ObjectiveTo develop an artificial intelligence (AI)–enabled electrocardiogram (ECG) algorithm capable of comprehensive, human-like ECG interpretation and compare its diagnostic performance against conventional ECG interpretation methods.MethodsWe developed a novel AI-enabled ECG (AI-ECG) algorithm capable of complete 12-lead ECG interpretation. It was trained on nearly 2.5 million standard 12-lead ECGs from over 720,000 adult patients obtained at the Mayo Clinic ECG laboratory between 2007 and 2017. We then compared the need for human over-reading edits of the reports generated by the Marquette 12SL automated computer program, AI-ECG algorithm, and final clinical interpretations on 500 randomly selected ECGs from 500 patients. In a blinded fashion, 3 cardiac electrophysiologists adjudicated each interpretation as (1) ideal (ie, no changes needed), (2) acceptable (ie, minor edits needed), or (3) unacceptable (ie, major edits needed).ResultsCardiologists determined that on average 202 (13.5%), 123 (8.2%), and 90 (6.0%) of the interpretations required major edits from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively. They considered 958 (63.9%), 1058 (70.5%), and 1118 (74.5%) interpretations as ideal from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively. They considered 340 (22.7%), 319 (21.3%), and 292 (19.5%) interpretations as acceptable from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively.ConclusionAn AI-ECG algorithm outperforms an existing standard automated computer program and better approximates expert over-read for comprehensive 12-lead ECG interpretation. To develop an artificial intelligence (AI)–enabled electrocardiogram (ECG) algorithm capable of comprehensive, human-like ECG interpretation and compare its diagnostic performance against conventional ECG interpretation methods. We developed a novel AI-enabled ECG (AI-ECG) algorithm capable of complete 12-lead ECG interpretation. It was trained on nearly 2.5 million standard 12-lead ECGs from over 720,000 adult patients obtained at the Mayo Clinic ECG laboratory between 2007 and 2017. We then compared the need for human over-reading edits of the reports generated by the Marquette 12SL automated computer program, AI-ECG algorithm, and final clinical interpretations on 500 randomly selected ECGs from 500 patients. In a blinded fashion, 3 cardiac electrophysiologists adjudicated each interpretation as (1) ideal (ie, no changes needed), (2) acceptable (ie, minor edits needed), or (3) unacceptable (ie, major edits needed). Cardiologists determined that on average 202 (13.5%), 123 (8.2%), and 90 (6.0%) of the interpretations required major edits from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively. They considered 958 (63.9%), 1058 (70.5%), and 1118 (74.5%) interpretations as ideal from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively. They considered 340 (22.7%), 319 (21.3%), and 292 (19.5%) interpretations as acceptable from the computer program, AI-ECG algorithm, and final clinical interpretations, respectively. An AI-ECG algorithm outperforms an existing standard automated computer program and better approximates expert over-read for comprehensive 12-lead ECG interpretation.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Kashou et al. (2021) conducted an observational in Adults requiring standard 12-lead ECG (n=500). AI-ECG algorithm vs. Marquette 12SL automated computer program was evaluated on Mean interpreter composite score for diagnostic accuracy (1=ideal, 2=acceptable, 3=unacceptable) (p=<0.0001). The AI-ECG algorithm improved diagnostic accuracy compared to the Marquette 12SL automated computer program, reducing the mean interpreter composite score from 1.497 to 1.377 (p<0.0001).

synapsesocial.com/papers/6a372a8cd9e0d8cad3c75b5a https://doi.org/10.1016/j.cvdhj.2021.04.002

View Full Paper

Ask AI

Bookmark

Share

An artificial intelligence–enabled ECG algorithm for comprehensive ECG interpretation: Can it pass the ‘Turing test’?

Figures

Key Points

Study Design

Structured PICO

Main Result

Limitations

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

An artificial intelligence–enabled ECG algorithm for comprehensive ECG interpretation: Can it pass the ‘Turing test’?

Figures

Key Points

Study Design

Structured PICO

Main Result

Limitations

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading