Evaluating OSCE workflow designs: impacts of grader type, modality, and exam timing

To examine how grading workflows and evaluator backgrounds affect student performance in a multi-station objective structured clinical examination (OSCE) at a school of pharmacy. The study compares scores by faculty department (Pharmacy Practice vs. others), grading method (in-person vs. remote asynchronous), and exam session timing. Sixty-six students completed six OSCE stations, and their performances were assessed using both analytical and global rubrics by faculty graders from various departments, either in-person or remotely. Statistical analysis was conducted using linear mixed-effects models to evaluate the effects of grader background, grading method, and exam timing on student scores. Grader background and grading method significantly impacted student OSCE scores—Pharmacy Practice faculty and remote graders tended to give lower scores. Exam timing had no significant effect. Asynchronous remote grading was associated with lower OSCE scores than live grading, suggesting that grading modality may meaningfully influence scoring patterns. Together with differences by evaluator background, these findings support the need for targeted rater calibration and standardized scoring practices to promote fair and reliable OSCE assessment in pharmacy education.