BACKGROUND: Competency-Based Medical Education (CBME) relies on frequent, competency-focused assessments, which can be challenging to implement consistently. Artificial Intelligence (AI) holds promise to improve assessment efficiency, objectivity, and feedback in CBME, but its use remains in early stages with limited understanding of current practices and evaluation methods. This study aims to map existing AI applications in CBME assessments to guide future work. METHODS: A comprehensive search was performed in MEDLINE (Ovid), EMBASE (Ovid), PsycINFO, and Scopus using tailored keywords and MeSH terms. Included studies focused on the deployment of AI for assessment within CBME, covering applications in generating, analyzing, or interpreting evaluation data across undergraduate, graduate, and continuing professional education. The PRISMA-ScR guidelines were used to ensure transparent reporting, and findings were synthesized following Levac et al.'s approach. RESULTS: Of the 1002 search results, 32 studies met the inclusion criteria. Key findings indicate a wide application of AI from surgical or procedural skill assessment, to clinical note assessment, communication assessment, feedback generation, projected trainee performance, and analysis of narrative feedback from supervisors. CONCLUSION: This review highlights potential advantages, such as timely evaluations, and challenges, such as lack of granularity, of AI integration. In conclusion, thoughtful integration of AI into competency-based medical education can complement traditional assessment methods and enhance learner outcomes, provided it is supported by robust infrastructure, ethical oversight, and collaborative policy development.
Hui et al. (2025) studied this question.