The Predictive Performance of General Surgery Milestones on Post-Graduation Outcomes

Objective: To establish whether Accreditation Council for Graduate Medical Education Milestones predict future performance of general surgery trainees. Summary Background Data: Milestones provide bi-annual assessments of trainee progress across six competencies. It is unknown whether the Milestones predict surgeon performance after the transition to independent practice. Methods: We performed a retrospective cohort study of surgeons with complete Milestone assessments in the fourth and fifth clinical years who treated patients in acute care hospitals within Florida, New York, and Pennsylvania, 2015-2018. To account for the multiple ways in which the Milestone assessments might predict post-graduation outcomes, we included 120 Milestones features in our elastic net machine learning models. The primary outcome was risk-adjusted patient death or serious morbidity. Results: 278 general surgeons were included in the study. Milestone assessments 6-months into the fourth clinical year displayed a normal score distribution while multicollinearity and low score discrimination at the final assessment period were detected. Individual Milestones features from the Patient Care, Professionalism, and Systems-based Practice domains were most predictive of patient-related outcomes. For example, surgeons with worse patient outcomes had significantly lower scores in Patient Care 3 when compared to surgeons with better patient outcomes (High DSM, yes: 2.86 vs. no: 3.04, P =0.011). Conclusions: The Milestones features that were most predictive of better patient outcomes related to intraoperative skills, ethical principles, and patient navigation and safety, measured 12-18 months prior to graduation. The development of a parsimonious set of evidence-based Milestones that better correlate with surgeon experience could enhance surgical education