August 1, 2019The Lancet
An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction
Key result
AI-enabled ECG identifies atrial fibrillation from normal sinus rhythm with an AUC of ~0.87.
- AUC 0.87
- 95% CI 0.86-0.88
- n=180,922
Why the study?
Atrial fibrillation is frequently asymptomatic and underdetected, and existing screening methods require prolonged monitoring with limited yield and high cost.
Does an artificial intelligence-enabled ECG algorithm identify patients with atrial fibrillation during normal sinus rhythm?
Key Points
- To develop and validate a convolutional neural network that detects the electrocardiographic signature of atrial fibrillation from standard 10-second, 12-lead ECGs recorded during normal sinus rhythm.
- Analyzed 649,931 digital normal sinus rhythm 12-lead ECGs from 180,922 adult patients treated at the Mayo Clinic ECG laboratory between 1993 and 2017.
- Allocated ECGs to training (454,789 ECGs from 126,526 patients), internal validation (64,340 ECGs from 18,116 patients), and testing (130,802 ECGs from 36,280 patients) datasets in a 7:1:2 ratio.
- Trained a convolutional neural network to classify patients with prior verified atrial fibrillation or flutter, evaluating performance on the testing dataset using ROC analysis.
- In the testing dataset, where 3,051 (8.4%) of 36,280 patients had prior verified atrial fibrillation, a single sinus-rhythm ECG predicted atrial fibrillation with an AUC of 0.87 (95% CI 0.86–0.88), sensitivity of 79.0% (95% CI 77.5–80.4), specificity of 79.5% (95% CI 79.0–79.9), and overall accuracy of 79.4% (95% CI 79.0–79.9).
- Analyzing all ECGs acquired within the first month of interest increased performance to an AUC of 0.90 (95% CI 0.90–0.91), sensitivity of 82.3% (95% CI 80.9–83.6), specificity of 83.4% (95% CI 83.0–83.8), and overall accuracy of 83.3% (95% CI 83.0–83.7).
Study Design
Type
Observational (n=180,922)
Multicenter
No
Structured PICO
Does an artificial intelligence-enabled ECG algorithm identify patients with atrial fibrillation during normal sinus rhythm?
P
Population
180,922 adult patients with 649,931 normal sinus rhythm ECGs used to train and validate an AI algorithm to identify atrial fibrillation.
E
Exposure
Artificial intelligence (AI)-enabled electrocardiograph (ECG) using a convolutional neural network to detect the electrocardiographic signature of atrial fibrillation present during standard 10-second, 12-lead normal sinus rhythm ECGs.
C
Comparator
Clinical diagnosis of atrial fibrillation (verified rhythm labels by trained personnel under cardiologist supervision).
O
Outcome
Model performance evaluated by area under the curve (AUC), accuracy, sensitivity, specificity, and F1 score.surrogate
Main Result
Effect estimate: AUC 0.87 (95% CI 0.86-0.88)
Key expert perspectives
Captured external expert commentary on this paper, strongest first. Original sources are linked where available.
“This study used patients who were already under investigation for atrial fibrillation for the testing and algorithm input dataset. This really limits the applicability of the findings to real world situations, where we would like to use ECGs to determine which patients will benefit from certain treatment for AF to decrease their risks of having a stroke.”
“Some quick calculations about the algorithm in its tested settings finds that about 10% of the patients in the dataset the algorithm was tested on are sick, which is a lot higher than the proportion of those with AF in the general population. When the algorithm identifies AF, it's a false result 75% of time. In healthy patients, the algorithm detected 20% of cases with AF and in sick patients (those known to have AF), it detected AF 80% of the time. However, since there were a lot of healthy patients, there will have been a lot of false alarms, perhaps too many for the algorithm to be useful right away.”
FJFrank J. WoltersEpidemiologist, Erasmus University Medical Center
“The authors compared ECGs of patients with atrial fibrillation to those of relatively young controls who did not have atrial fibrillation during the 24-year study period. This approach led to selection of extremely healthy controls with ECGs that are very different from patients in the average transient ischaemic attack or stroke clinic.”
Implication
An AI-enabled convolutional neural network can detect the electrocardiographic signature of atrial fibrillation from standard 10-second, 12-lead ECGs acquired during normal sinus rhythm, enabling rapid point-of-care screening.
Abstract
BACKGROUND Atrial fibrillation is frequently asymptomatic and thus underdetected but is associated with stroke, heart failure, and death. Existing screening methods require prolonged monitoring and are limited by cost and low yield. We aimed to develop a rapid, inexpensive, point-of-care means of identifying patients with atrial fibrillation using machine learning. METHODS We developed an artificial intelligence (AI)-enabled electrocardiograph (ECG) using a convolutional neural network to detect the electrocardiographic signature of atrial fibrillation present during normal sinus rhythm using standard 10-second, 12-lead ECGs. We included all patients aged 18 years or older with at least one digital, normal sinus rhythm, standard 10-second, 12-lead ECG acquired in the supine position at the Mayo Clinic ECG laboratory between Dec 31, 1993, and July 21, 2017, with rhythm labels validated by trained personnel under cardiologist supervision. We classified patients with at least one ECG with a rhythm of atrial fibrillation or atrial flutter as positive for atrial fibrillation. We allocated ECGs to the training, internal validation, and testing datasets in a 7:1:2 ratio. We calculated the area under the curve (AUC) of the receiver operatoring characteristic curve for the internal validation dataset to select a probability threshold, which we applied to the testing dataset. We evaluated model performance on the testing dataset by calculating the AUC and the accuracy, sensitivity, specificity, and F1 score with two-sided 95% CIs. FINDINGS We included 180 922 patients with 649 931 normal sinus rhythm ECGs for analysis: 454 789 ECGs recorded from 126 526 patients in the training dataset, 64 340 ECGs from 18 116 patients in the internal validation dataset, and 130 802 ECGs from 36 280 patients in the testing dataset. 3051 (8·4%) patients in the testing dataset had verified atrial fibrillation before the normal sinus rhythm ECG tested by the model. A single AI-enabled ECG identified atrial fibrillation with an AUC of 0·87 (95% CI 0·86-0·88), sensitivity of 79·0% (77·5-80·4), specificity of 79·5% (79·0-79·9), F1 score of 39·2% (38·1-40·3), and overall accuracy of 79·4% (79·0-79·9). Including all ECGs acquired during the first month of each patient's window of interest (ie, the study start date or 31 days before the first recorded atrial fibrillation ECG) increased the AUC to 0·90 (0·90-0·91), sensitivity to 82·3% (80·9-83·6), specificity to 83·4% (83·0-83·8), F1 score to 45·4% (44·2-46·5), and overall accuracy to 83·3% (83·0-83·7). INTERPRETATION An AI-enabled ECG acquired during normal sinus rhythm permits identification at point of care of individuals with atrial fibrillation. FUNDING None.
Discussion
No takes yet. Share an insight, caveat, or question.
Cite This Study
Attia et al. (2019) conducted an observational in Atrial fibrillation (n=180,922). Artificial intelligence (AI)-enabled electrocardiograph (ECG) algorithm was evaluated on Identification of atrial fibrillation from a normal sinus rhythm ECG (AUC 0.87, 95% CI 0.86-0.88). An AI-enabled ECG algorithm applied to standard normal sinus rhythm ECGs identified individuals with atrial fibrillation with an AUC of 0.87 (95% CI 0.86-0.88).