A deep learning-based CMR segmentation model exhibited significant racial bias, achieving Dice scores of approximately 94% in white subjects compared to 86-89% in minority ethnic groups.
Cross-Sectional (n=5,903)
Yes
Does a deep learning-based cine CMR segmentation model exhibit sex or racial bias in performance among subjects from the UK Biobank?
Deep learning models for CMR segmentation can exhibit significant racial bias due to imbalanced training data, highlighting the need for diverse datasets in AI development.
Absolute Event Rate: 94% vs 86%
p-value: p=<0.001
Abstract Background Artificial intelligence (AI) techniques have been proposed for automation of cine CMR segmentation for functional quantification. However, in other applications AI models have been shown to have potential for sex and/or racial bias. Objectives To perform the first analysis of sex/racial bias in AI-based cine CMR segmentation using a large-scale database. Methods A state-of-the-art deep learning (DL) model was used for automatic segmentation of both ventricles and the myocardium from cine short-axis CMR. The dataset consisted of end-diastole and end-systole short-axis cine CMR images of 5,903 subjects from the UK Biobank database (61.5±7.1 years, 52% male, 81% white). To assess sex and racial bias, we compared Dice scores and errors in measurements of biventricular volumes and function between patients grouped by race and sex. To investigate whether segmentation bias could be explained by potential confounders, a multivariate linear regression and ANCOVA were performed. Results We found statistically significant differences in Dice scores (white ∼94% vs minority ethnic groups 86-89%) as well as in absolute/relative errors in volumetric and functional measures, showing that the AI model was biased against minority racial groups, even after correction for possible confounders. Conclusions We have shown that racial bias can exist in DL-based cine CMR segmentation models. We believe that this bias is due to the imbalanced nature of the training data (combined with physiological differences). This is supported by the results which show racial bias but not sex bias when trained using the UK Biobank database, which is sex-balanced but not race-balanced. Condensed Abstract AI algorithms have the potential to reflect or exacerbate racial/sex disparities in healthcare. We aimed to determine the impact of sex and race on the performance of an AI segmentation model for automatic CMR quantification in a cohort of 5,903 subjects from the UK Biobank database, which is sex-balanced but not race-balanced. We tested the model’s bias in performance using Dice scores and absolute/relative errors in measurements of biventricular volumes and function. Our study demonstrates that the model had a racial bias but no sex bias, and that subject characteristics and co-morbidities could not explain this bias.
Puyol‐Antón et al. (Thu,) conducted a cross-sectional in Healthy subjects (assessing AI segmentation bias) (n=5,903). Deep learning-based CMR segmentation (nnU-Net) vs. Comparison between racial and sex groups was evaluated on Dice similarity coefficient (DSC) (p=<0.001). A deep learning-based CMR segmentation model exhibited significant racial bias, achieving Dice scores of approximately 94% in white subjects compared to 86-89% in minority ethnic groups.