Key result
The ArNet2 deep learning algorithm achieved an F1 score of 0.95 for atrial fibrillation detection, significantly outperforming the ArNet model (F1 0.93, p<0.001) and generalizing well across geographies, ages, and sexes.
Why the study?
Deep learning algorithms require evaluation across distribution shifts to ensure robust generalization performance across geography, ages, and sexes.
Does the ArNet2 deep learning model accurately detect atrial fibrillation events from continuous ECG across different geographies, ages, and sexes compared to benchmark models?
Observational (n=4,048)
Yes
Does the ArNet2 deep learning model accurately detect atrial fibrillation events from continuous ECG across different geographies, ages, and sexes compared to benchmark models?
Absolute Event Rate: 0.95% vs 0.93%
p-value: p=<0.001
The ArNet2 deep learning model provides robust, generalizable detection of atrial fibrillation from continuous ECG across diverse demographic and geographic populations.
Supports generalizable AF detection across populations; extends benchmarks yet leaves open prospective validation before adoption.
To drive health innovation that meets the needs of all and democratize healthcare, there is a need to assess the generalization performance of deep learning (DL) algorithms across various distribution shifts to ensure that these algorithms are robust. This retrospective study is, to the best of our knowledge, an original attempt to develop and assess the generalization performance of a DL model for AF events detection from long term beat-to-beat intervals across geography, ages and sexes. The new recurrent DL model, denoted ArNet2, is developed on a large retrospective dataset of 2,147 patients totaling 51,386 h obtained from continuous electrocardiogram (ECG). The model's generalization is evaluated on manually annotated test sets from four centers (USA, Israel, Japan and China) totaling 402 patients. The model is further validated on a retrospective dataset of 1,825 consecutives Holter recordings from Israel. The model outperforms benchmark state-of-the-art models and generalized well across geography, ages and sexes. For the task of event detection ArNet2 performance was higher for female than male, higher for young adults (less than 61 years old) than other age groups and across geography. Finally, ArNet2 shows better performance for the test sets from the USA and China. The main finding explaining these variations is an impairment in performance in groups with a higher prevalence of atrial flutter (AFL). Our findings on the relative performance of ArNet2 across groups may have clinical implications on the choice of the preferred AF examination method to use relative to the group of interest.
No takes yet. Share an insight, caveat, or question.
Biton et al. (2023) conducted an observational in Atrial fibrillation (n=4,048). ArNet2 deep learning algorithm vs. ArNet and XGB models was evaluated on F1 score for 60-beat-window classification of atrial fibrillation on the UVAF-test set (p=<0.001). The ArNet2 deep learning algorithm achieved an F1 score of 0.95 for atrial fibrillation detection, significantly outperforming the ArNet model (F1 0.93, p<0.001) and generalizing well across geographies, ages, and sexes.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: