The assessment of clinical competence in medical students during the COVID-19 pandemic: a scoping review

BACKGROUND: The COVID-19 pandemic necessitated unprecedented changes to clinical education and assessment. Restrictions on in-person clinical encounters led to rapid adoption of digital assessment modalities, raising concerns about graduate preparedness and the validity of pandemic-adapted assessment approaches. Wojniusz, [40]. OBJECTIVES: This scoping review aimed to: (1) map approaches to assessing clinical competence in medical students during COVID-19; and (2) evaluate the validity, reliability, feasibility, and acceptability of pandemic-adapted assessments. METHODS: Following Arksey and O'Malley's framework, we systematically searched six databases (PubMed/MEDLINE, Scopus, Web of Science, EMBASE, ERIC, Google Scholar) for peer-reviewed empirical studies published March 2020-March 2023 examining clinical competence assessment in final-year medical students. Studies were analyzed using Van der Vleuten's utility framework (validity, reliability, feasibility, acceptability) and Miller's Pyramid to categorize competency levels assessed. RESULTS: From 1,247 references, 13 studies met inclusion criteria, representing 8 countries. Virtual OSCEs were the predominant assessment method (n = 8, 62%). Study designs were predominantly weak: surveys (n = 6), evaluative case studies (n = 4), and observational studies (n = 2), with only three cohort studies providing stronger evidence. Most studies (92%) focused on feasibility and acceptability rather than validity or psychometric properties. Physical examination and procedural skills assessment remained a critical unresolved limitation across all digital modalities. No studies employed generalizability theory or Kane's validity framework. CONCLUSIONS: A paucity of rigorous research on pandemic-adapted assessment validity exists. Published studies employed weak designs producing non-generalizable findings focused on feasibility over validity. Urgent research priorities include: longitudinal cohort studies comparing pandemic-trained graduates to pre-pandemic cohorts; psychometric validation of hybrid assessment models now institutionalized; application of contemporary validity frameworks to digital assessments; and equity analysis of differential impacts across student populations. The field requires systematic validity evidence before permanently adopting pandemic-adapted approaches.