Abstract Purpose Quality feedback is essential for the training and development of surgeons. Coupled with this, medical education is experiencing a paradigm shift towards competency-based training. The successful implementation of the Entrustable Professional Activities (EPAs) assessment framework into many surgical training residency and fellowship programs allows for a broad focus on utilizing not only quality feedback but also trainees’ self-reflections as tools to ensure readiness for autonomous practice. This study comprehensively investigates the quality of EPA feedback and self-reflections for General Surgery (GS) EPAs implemented within a GS residency program. Methods A mobile application, SEPA, designed by a Midwestern academic department of surgery was used by the residents and faculty members from the department at a single tertiary care center. Entrustment levels and free text feedback were collected from July 2018 to November 2021 from four GS EPAs, right lower quadrant pain, gallbladder disease, inguinal hernia repair, and care for a traumatically injured patient. Feedback was first manually coded for the inclusion or exclusion of specific features. We then applied a previously developed natural language processing (NLP) model that was validated using SIMPL OR narrative feedback to generate probabilities that an instance of feedback in SEPA could be classified as “high quality.” Manual coding was used for comparison. Predicted probabilities were then analyzed by EPA features, including specific procedure, phase of procedure, and trainee autonomy level. Results The quality of feedback and self-reflection did not vary between faculty and resident ratings. However, feedback quality did differ based on perioperative stage for which feedback was given and the level of autonomy at which the resident was assessed. Intraoperative feedback was of higher quality than pre- and post-operative feedback. Feedback based on autonomy level showed a rainbow shaped curve as shown in Fig. 1, with the lowest and highest autonomy levels receiving the lowest quality feedback. Conclusions To improve feedback to resident trainees, this study’s results suggest focusing on pre- and post-operative feedback as well as feedback given to trainees assessed at the lowest and highest levels of autonomy.
Sundar et al. (Thu,) studied this question.