Evaluation of the objective structured clinical examination in the assessment of nuclear medicine residency training

Abstract Objectives We evaluated the capacity of the Objective Structured Clinical Examination (OSCE) to certify resident competency in nuclear medicine (NM). Materials and methods A retrospective analysis was conducted on the OSCE scores of residents who participated in the examination for the certification of NM physicians in Beijing from 2019 to 2024, with a focus on difficulty, discrimination, reliability, and validity. Results The OSCE is comprised of six stations, including high-radioactivity room operations, imaging room operations, PET/CT case analysis, SPECT/CT case analysis, patient interviews and report writing. The mean total OSCE scores (mean ± SD) were 88.58 ± 2.37 points (2019), 89.13 ± 2.72 points (2020), 86.15 ± 2.75 points (2021), 92.96 ± 2.31 points (2022), 91.11 ± 1.37 points (2023), and 89.08 ± 2.75 points (2024), respectively. The overall difficulty coefficients of the OSCE were 0.89, 0.89, 0.86, 0.88, 0.91, and 0.89, respectively. The internal consistency reliability values (Cronbach’s α) were 0.66, 0.76, 0.57, 0.28, 0.05, and 0.69, respectively, across the years. The station-level discrimination indices ranged from 0.46 to 0.83. Among the 30 stations that were double-scored by two examiners, 73.33% (22/30) of the stations demonstrated excellent interrater consistency. Conclusion The OSCE effectively simulates clinical environments and provides a comprehensive assessment of the clinical competence of NM residents. Critical relevance statement This study conducted a multidimensional evaluation of 6-year OSCE data, focusing on difficulty, discrimination, reliability, and validity, demonstrating its effectiveness in simulating clinical environments and assessing the clinical competence of NM residents. Key Points Compared to the numerous studies on medical students, relatively few studies on the OSCE target NM residents. The OSCE exhibited acceptable station difficulty and discrimination, moderate reliability, and satisfactory content validity for assessing NM residents.