An objective structured clinical examination for evaluating psychiatric clinical clerks

PURPOSE: To assess the feasibility, reliability, and validity of an objective structured clinical examination (OSCE) for psychiatric clinical clerks. METHOD: In 1995 two parallel forms of a ten-station OSCE (eight clinical stations, two writing stations) were developed at the University of Toronto Faculty of Medicine Each 12-minute performance-based clinical station was assessed by a faculty psychiatrist using both a checklist for each student's performance content and a global-rating scale of the performance process. The students' clinical-station scores were calculated as the average of their content and process scores (expressed as percentages). Examiners also recorded an overall judgment of each students' performance (pass, borderline, or fail) and wrote in collaboration with the standardized patient (SP) at that station comments on each student's performance. There were two criteria for a passing grade: a total mark of 60% or higher across all ten stations and a "pass" or "borderline" mark in at least five of the eight clinical stations. Each OSCE form was administered three times. RESULTS: The first form was used to examine 94 clerks, the second form to examine 98 clerks. The students' mean scores for the two forms were 70. 47% (SD, 6. 33%) and 67. 66% (SD, 7. 05%), respectively. In addition to the standard evaluation information collected on the students, several critical incidents occurred (e. g. , a student's loss of control of emotions) that may identify potential problems in professional conduct. The direct cost for one administration of the examination was approximately Can3, 300: the largest portion of this was for the SPs' time spent in training and performing their roles. CONCLUSION: Preliminary evidence suggests that a psychiatry OSCE is feasible for assessing complex psychiatric skills. However, careful attention must be paid to SP training, examination monitoring, detection of critical incidents, and provision of feedback to students, faculty, and SPs. The university's previous system of oral examinations required approximately 600 faculty hours per year. The OSCE requires approximately 450 faculty hours, and the 150 hours saved almost cover the Can20, 000 that the examination costs each year. In all, the OSCE is an evaluation system that has demonstrable reliability and is more enjoyable for both the faculty and the students.