Key result
AI-ECG model predicts incident HF less accurately in young Black vs White patients, AUC 0.69 vs 0.80.
Why the study?
Demographic biases in the performance of deep learning models predicting incident heart failure risk had not been well-studied.
Observational (n=326,518)
No
Absolute Event Rate: 0.69% vs 0.8%
Deep learning models predicting incident heart failure from ECGs exhibit significant demographic biases, particularly underperforming in young Black women, highlighting the need for individualized probability thresholds to mitigate algorithmic disparities.
ECG HF prediction models warrant caution in young Black patients; leaves open whether individualized thresholds improve equity without prospective validation.
BACKGROUND: Deep learning models may combat widening racial disparities in heart failure outcomes through early identification of individuals at high risk. However, demographic biases in the performance of these models have not been well-studied. METHODS: This retrospective analysis used 12-lead ECGs taken between 2008 and 2018 from 326 518 patient encounters referred for standard clinical indications to Stanford Hospital. The primary model was a convolutional neural network model trained to predict incident heart failure within 5 years. Biases were evaluated on the testing set (160 312 ECGs) using the area under the receiver operating characteristic curve, stratified across the protected attributes of race, ethnicity, age, and sex. RESULTS: There were 59 817 cases of incident heart failure observed within 5 years of ECG collection. The performance of the primary model declined with age. There were no significant differences observed between racial groups overall. However, the primary model performed significantly worse in Black patients aged 0 to 40 years compared with all other racial groups in this age group, with differences most pronounced among young Black women. Disparities in model performance did not improve with the integration of race, ethnicity, sex, and age into model architecture, by training separate models for each racial group, or by providing the model with a data set of equal racial representation. Using probability thresholds individualized for race, age, and sex offered substantial improvements in F1 scores. CONCLUSIONS: The biases found in this study warrant caution against perpetuating disparities through the development of machine learning tools for the prognosis and management of heart failure. Customizing the application of these models by using probability thresholds individualized by race, ethnicity, age, and sex may offer an avenue to mitigate existing algorithmic disparities.
No takes yet. Share an insight, caveat, or question.
Kaur et al. (2023) conducted an observational in Incident heart failure (n=326,518). Deep learning ECG model vs. Performance across demographic subgroups was evaluated on Area under the receiver operating characteristic curve (AUC) for predicting incident heart failure within 5 years in Black patients aged 0-40 years vs Non-Hispanic White patients aged 0-40 years. A deep learning model predicting incident heart failure from ECGs performed significantly worse in Black patients aged 0 to 40 years (AUC 0.69) compared to Non-Hispanic White patients in the same age group (AUC 0.80), with disparities most pronounced in young Black women.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: