Psychometric characteristics of Korean Medical Licensing Examination items from 2012 to 2022 analyzed using the Rasch model and 2-parameter logistic model

PURPOSE: This study characterized the psychometric properties of Korean Medical Licensing Examination items administered from 2012 to 2022 using the Rasch and 2-parameter logistic models and descriptively examined items classified as highly difficult. METHODS: Item parameters were estimated separately for each examination year using the Rasch and 2-parameter logistic models implemented in the irtQ package in R. Descriptive statistics and correlations between model-based difficulty estimates were calculated. Items with a 2-parameter logistic difficulty estimate of b ≥2.0 were subjected to content analysis. RESULTS: Correlations between Rasch and 2-parameter logistic item-difficulty estimates ranged from 0.70 to 0.80 across examination years. Under the 2-parameter logistic model, the proportion of items with b ≥2.0 ranged from 4.7% to 8.9% and showed no monotonic temporal trend; the proportions were 5.0% in 2021 and 5.6% in 2022. The Rasch model generally produced smaller estimated standard errors than the 2-parameter logistic model. CONCLUSION: Rasch and 2-parameter logistic difficulty estimates showed strongly correlated rankings, although correlation alone did not establish agreement between their absolute estimates. The Rasch model demonstrated greater numerical stability under the present calibration conditions, but model selection should also consider model fit, test information, and the intended use of scores. Because examination years were calibrated separately and no pre-disclosure or control condition was available, the observed annual differences cannot be interpreted as evidence that item disclosure caused changes in item difficulty or discrimination.