Assessing Medical Student Lifestyle Medicine Skills Using an Objective Structured Clinical Examination

(1) The purpose of this project was to create and collect validity evidence for a lifestyle medicine objective structured clinical examination (OSCE) station to assess medical students’ performance related to lifestyle medicine competencies. (2) We developed a lifestyle medicine case/station with an associated observation checklist and rubric. We piloted the checklist and rubric in one lifestyle medicine OSCE station, securing triplicate scores of each student’s performance. For analysis, generalizability (G) theory was utilized for observation checklist data and interclass correlation coefficients (ICC) for patient encounter notes (PENs). (3) One hundred and fifteen third-year medical students completed the lifestyle medicine OSCE station in the Internal and Family Medicine Clerkship. The generalizability coefficient and Phi-coefficient based on the number of encounters (P = 115), facet 1 (nfacet1 = 10 assessment tool checklist items), and facet 2 (nfacet2 = two performance ratings in the live examination) were 0.71 and 0.69, respectively. The average interclass correlation coefficient (ICC) measure for PEN was 0.79 (CI = 0.69–0.85). (4) For this OSCE station, the G-coefficient provides positive indicators for the validity of the observation checklist items. Similarly, the ICC result provides validity evidence for the usefulness of the PEN rubric for capturing lifestyle medicine knowledge reflected in students’ PEN notes.