Unpacking the dark variance of differential attainment on examinations in overseas graduates

CONTEXT: Differential performance in postgraduate examinations between home medical graduates and those who qualified outside their country of practice is well recognised. This difference is especially marked in the practical component of the UK Membership of the Royal College of General Practitioners (MRCGP) examination. The potential causes of such disparities are not well understood. METHODS: Data were available for 1874 international medical graduates who applied for general practice (GP) specialty training in the UK in 2008-2012. The primary outcome was performance in the Clinical Skills Assessment (CSA) OSCE component of the MRCGP. The main predictors were performance on a situational judgement test (SJT) and clinical problem-solving test (CPST), a test of applied clinical knowledge, used in the selection for GP training. Data relating to demographic characteristics and English language fluency were also available. To better understand the relationship between the predictors, the selection measures and the outcome, a series of univariable and multivariable models were developed and tested, concluding with a structural equation model to explore causality. RESULTS: The CSA rating was more strongly predicted by SJT scores (standardised beta, 0.26) than by performance on the CPST (standardised beta, 0.17). There was a relationship between English language fluency and CSA score that was mainly mediated via SJT performance. CONCLUSIONS: These findings demonstrate that performance on an SJT predicts performance in a high-fidelity clinical simulation (the CSA) in international medical graduates. Although the constructs tested by SJTs are debated, and are likely to vary across settings, culturally appropriate knowledge of interpersonal competence is likely to be evaluated. Improving the confidence of doctors in this area through targeted educational interventions, rather than focusing on increased clinical knowledge, is likely to be more effective at reducing disparities observed in postgraduate examination performance. Thus, there are important implications for the design of specialty selection and licensing assessments globally.