ABSTRACT Background Pediatric Emergency Medicine (PEM) fellowship programs use Milestones to assess fellows, but validity evidence for these ratings is needed. Objectives (1) Define benchmark levels of competence for the 6 Milestones competency domains and 23 competencies. (2) Determine if Milestones ratings at graduation predict passing the American Board of Pediatrics (ABP) certification exam. Methods This multi‐site retrospective cross‐sectional study included all PEM fellows (2014–2022) from 80 programs in the U.S. who took the ABP certification exam. Milestones ratings from the Accreditation Council for Graduate Medical Education (ACGME) were provided to the Association of Pediatric Program Directors and analyzed. Logistic mixed effects regression models assessed associations between American Board of Pediatrics (ABP) exam pass rates and Milestones at graduation. Results Data from 1302 fellows (1171 from 3‐year and 131 2‐year graduates) yielded 90,229 Milestones ratings from which benchmarks were established. Systems‐based practice consistently had the lowest mean across all fellowship years. 3‐year graduates had significantly higher Milestones at graduation compared to 2‐year graduates ( p < 0.001). 3‐year graduates with ABP certification had significantly higher graduating mean Milestone ratings averaged over all domains compared to those without evidence of certification ( p = 0.003), but 2‐year graduates had no significant difference ( p = 0.78). Conclusion This national study of PEM fellows establishes validity evidence for use of year‐end Milestones ratings from fellowship within all domains as a potential predictor for achieving ABP certification exam. Fellows from 3‐year programs had higher mean Milestones at graduation and higher certification exam pass rates.
Wilson et al. (Wed,) studied this question.