Using a standardised patient assessment to measure professional attributes

INTRODUCTION: Professionalism is an important topic in medical education today. While much work has focused on defining professionalism and teaching medical students the appropriate interpersonal behaviours, relatively little research has looked at meaningful ways of assessing the relevant attributes. METHOD: The Educational Commission for Foreign Medical Graduates (ECFMG) clinical skills assessment (CSA) uses standardised patients (SPs) to evaluate the readiness of graduates of international medical schools to enter accredited graduate training programmes in the USA. Doctor interpersonal skills, including professional qualities such as rapport, are evaluated as part of the CSA. Attentiveness, attitude and empathy, all facets of professional behaviour, are specifically targeted as part of the assessment. RESULTS: To date, over 35 000 candidates have been assessed, encompassing more than 370 000 individual SP encounters. Based on a 1-year cohort of examinees, the reliability of the individual professionalism-related component scores ranged from 0.61 to 0.70. Doctors who had graduated from medical school more recently, or were younger, generally obtained higher ratings. Professional qualities were only marginally related to measures of basic science and clinical science proficiency. Female candidates were rated significantly higher than male candidates on all dimensions. CONCLUSIONS: While some professional behaviours are probably best measured using formats such as surveys, self-assessment and critical incident techniques, certain aspects of the domain can be reliably and validly measured in SP examinations.