What type of study is this?

This is a Systematic Review study.

October 11, 2025Open Access

Gender and Racial Bias Unveiled: Clinical Artificial Intelligence (AI) and Machine Learning (ML) Algorithms are Fanning the Flames of Inequity

Puntos clave

Only 16% of clinical AI/ML models used publicly available datasets, raising concerns about transparency.
84% of global models failed to report racial composition, indicating severe demographic bias in training datasets.
U.S. models fared better, with only 56% reporting race and 77% reporting gender in their datasets.
The study advocates for standardized disclosure of demographic data to combat inequity and bias in AI.

Resumen

Abstract Background This study aims to advocate for the continued evaluation of published clinical Artificial Intelligence (AI) and Machine Learning (ML) studies, including the reporting of demographic information, specifically gender, racial composition, and geographic location. Models are often trained on data lacking representation across basic demographics, potentially leading to biased outputs and exacerbating health disparities. Previous research in drug and device development has demonstrated the dangers of underrepresenting women and minority populations. This study aimed to assess the extent to which published clinical AI/ML studies report demographic information, specifically gender and racial composition, in their training datasets. Methods A systematic review was conducted in accordance with PRISMA guidelines. The databases that were used in the study include Ovid MEDLINE, Embase, PsycINFO, Scopus, Web of Science, and the Cochrane Library, for clinical AI/ML studies with direct implications for patient care. Inclusion criteria required models to be clinically actionable and not pre-clinical or administrative in scope. Two independent reviewers screened and extracted data using Covidence software, with conflicts resolved by a third reviewer. Results Out of 390 studies included, 84% of global models did not report the racial composition of their training data, while 31% lacked gender data. U.S.-based models performed slightly better, with 56% reporting race and 77% reporting gender. Only 16% of all models utilized publicly available, non-proprietary datasets. Conclusions The low frequency of demographic disclosure and limited use of open data raise serious concerns about the transparency, generalizability, and fairness of clinical AI/ML models. Standardized reporting of gender and racial composition in training data is urgently needed to ensure ethical and equitable deployment of these technologies.

Leer artículo completoexternamente

Preguntar a la IA

Me gusta

Guardar

Ver artículo completo

Cite This Study

Otokiti et al. (Thu,) studied this question.

synapsesocial.com/papers/68e9b1b5ba7d64b6fc131eac https://doi.org/https://doi.org/10.1093/oodh/oqaf027

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

Preguntar a la IA

Me gusta

Guardar

Ver artículo completo