Answers are grounded in this paper and cite their sources.
The promise of artificial intelligence (AI) and machine learning (ML) for improving clinical outcomes has been marked by modest successes, but also many failures. Access to data and to advanced algorithms makes developing AI/ML tools relatively easyâbut the real challenge is demonstrating their clinical utility. Clinical predictive algorithms (CPAs)âalgorithms for diagnosis or prognosisâare failing to meet their potential for two key reasons. First, when developers assess the quality of CPAs, they tend to rely too heavily on area under the receiver operating characteristic curve (AUC), ignoring the prevalence of the characteristic of interest, which is critical to understanding the positive and negative predictive value of the CPA. We propose a more holistic approach to optimizing the development of a CPA, as has been taken with bioanalytical diagnostic tests and physiologic biomarkers for decades. Second, insufficient emphasis has been placed on performing rigorous clinical trials to quantify the benefits and risks of implementing a CPA. We propose a fit-for-purpose, sequential clinical development approach analogous to development of new medicinal products, biomarkers, and other complex interventions, in some cases overseen by regulatory agencies. With these two recommendations met, the benefits and risks of a CPA will be clearer, resulting in greater credibility, transparency, and utility.
No takes yet. Share an insight, caveat, or question.
Ruberg et al. (2023) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: