Clinical skills assessment: limitations to the introduction of an "OSCE" (Objective Structured Clinical Examination) in a traditional Brazilian medical school

CONTEXT: Assessment of clinical skills has a central role in medical education and the selection of suitable methods is highly relevant. The OSCE (Objective Structured Clinical Examination) is now established as one of the most valid, reliable and effective tests for the assessment of clinical skills. OBJECTIVE: To describe student and faculty perceptions of an OSCE introduced in a traditional Brazilian medical school. TYPE OF STUDY: Descriptive, semi-quantitative study. SETTING: Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo. PARTICIPANTS: 258 junior medical students finishing an introductory course on basic clinical skills and six faculty members deeply involved with the OSCE administration. PROCEDURES: Over a period of three consecutive years, student perceptions on the examination were evaluated using a structured questionnaire containing several five-point scales; faculty members' opinions were collected using a structured questionnaire plus a personal interview. MAIN MEASUREMENTS: Student satisfaction or dissatisfaction with aspects of OSCE administration and positive or negative opinions from faculty members. RESULTS: Students were comfortable with cases and tasks, but nearly half (48%) of them criticized organizational aspects of the OSCE. Substantial proportions of students reported difficulties with both time management (70%) and stress control (70%). Improvement of several aspects of exams reduced criticism of organization to a minority (5%) of students, but the proportions of students reporting difficulties with time management (40%) and stress control (75%) during the exam remained virtually unchanged. Faculty members acknowledged the accuracy of the OSCE, but criticized its limitations for assessing the integrated approach to patients and complained that the examination was remarkably time and effort-consuming. The educational impact of the OSCE was felt to be limited, since other faculty members did not respond to the communication of exam results. CONCLUSIONS: In addition to shortage of resources and organizational difficulties, local cultural aspects and the absence of a more favorable educational climate may hinder lasting improvements in assessment methods in traditional medical schools.