Assessing performance in final year medical students. Can a postgraduate measure be used in an undergraduate setting?

BACKGROUND: The transition between medical school and graduate performance should be a continuum. This study aimed to evaluate an assessment tool developed for practising doctors when applied to undergraduates. METHODS: A 12-item rating form was developed from that used for practising doctors by the Royal Australasian College of Physicians. Over a 2-year period, senior doctors, junior doctors and nurses completed the rating form on final year medical students. Some students completed self-assessments. We performed factor analysis and correlated scores between raters and attachments. Correlating ratings with concurrent traditional assessment results across the year tested construct validity. RESULTS: Ten forms per student were distributed for all 123 students and 856 were returned (70%). Internal consistency was very high. In all, 71.1% of the variance was accounted for by two factors (clinical skills and humanistic). This factor structure is unchanged when restricted to different raters and is the same as that noted previously when rating practising doctors. There were good correlations between raters (including self) and between attachments. Nurse ratings were reliable but nurses rated students significantly lower on humanistic qualities. Correlations with traditional assessments were high when all traditional assessments were combined. Women scored more favourably than men on humanistic qualities. CONCLUSION: A rating instrument for doctors in practice retains the same factor structure and a high degree of reliability and validity for senior medical students. Reliable ratings by nurses have implications for measures of collegiality and teamwork. We believe the instrument could be a useful outcome measure for medical programmes and employers.