Does direct observation improve medical students' clerkship experiences?

Context and setting Year 3 and 4 medical students rotate through various clinical clerkships to develop evaluation and patient care skills. However, medical students are often unobserved and are not directly assessed by their supervisors during patient interactions. In addition, preceptors may feel they have inadequate time or skills to provide direct observation and feedback to students. The Brief Structured Clinical Observation (BSCO) is an assessment tool supporting spontaneous and brief faculty observation and feedback sessions of medical students. During the BSCO, the student notifies his or her supervisor before initiating a patient assessment. Later, the supervisor enters the patient's room as an unobtrusive, non-participating observer. The preceptor exits the room after obtaining 1–3 focused feedback points, usually within 5 minutes. Prior to the student's presentation of the patient, the supervisor provides the student with feedback of the observed encounter. Why the idea was necessary Medical students at the University of California San Francisco (UCSF) evaluate their experiences on each clinical clerkship via a computerised evaluation system (E-Value). In 2002–03, the scores for most clerkships, including paediatrics, were lowest for direct observation and feedback. What was done In 2003, we developed and piloted a 4 × 6 BSCO card. During the 9-month pilot period, all 15 faculty and the 33 core paediatric clerkship students at the pilot site participated in a BSCO instructional session. The goal was to ensure that students obtained 2 BSCOs per week. Student and faculty experiences with the BSCO were evaluated using an anonymous survey. In addition, we compared E-Value scores for the clerkship (on a 5-point Likert scale) during the pilot with scores from the previous 9 months, using T-tests. Evaluation of results and impact All 33 students participated in the pilot and completed an average of 0.5 BSCOs per student per week (range 0.2–1.2). Most encounters (61%) occurred in the urgent care clinic and included a portion of the physical examination (68%). Twelve faculty members performed BSCOs, and 8 completed surveys. The faculty estimated that each BSCO took 5–10 minutes to complete. Faculty members noted a positive effect of the BSCO on their abilities to observe, evaluate and provide feedback to students, as well as on their overall evaluation of students and their satisfaction as clinical educators. A total of 23 students completed surveys. On average, they reported the BSCO had a positive impact on their clerkship experience and was valuable to their education. In all, 75% of students identified the busy clinical service as a barrier to using the BSCO and 57% cited preceptor unavailability. Student clerkship evaluations were completed by 33/33 students during the pilot and 30/31 during the control period. The mean evaluation scores for quality of faculty teaching, adequacy of observation, adequacy of feedback and physical examination findings instruction all trended upwards during the pilot period, compared with the control period (although none of these differences were significant). The overall clerkship score increased from 3.8 to 4.3 (P = 0.04). Faculty staff at the pilot site asked to continue using the BSCO, and we have begun plans to implement the BSCO at all UCSF paediatric clerkship sites.