Generating ACGME-constraint-aware surgery residency call schedule software with the assistance of agentic coding large language models

Background Manual creation of a surgery residency call schedule is a labor-intensive process that adds to the administrative burden of a chief resident while often producing suboptimal fairness and work distribution. While commercial scheduling software exists, many programs struggle with inflexibility, high costs, and an inability to accommodate program-specific constraints. Methods We developed a scheduling software platform using AI-assisted development tools (Codex, Claude Code), both commercially available agentic coding environments. The system applies a rule-based, weekend-first assignment hierarchy with randomized tie-breaking under a search budget fixed in advance, then performs iterative weekday assignment, accommodating vacation requests and discrete call-shift swapping. The validation roster modeled a single general surgery residency program: 55 residents (32 categorical and 2 preliminary general surgery residents spanning PGY 1 through PGY 6; 20 off-service rotators from other programs; and 1 fellow) distributed across 10 call pools. Three prespecified call-related ACGME constraints were enforced (at least 4 days off per 28-day period, call no more frequently than every third night averaged over 4 weeks, and no consecutive 24-hour call). The 80-hour weekly maximum and the 24-hour period free of all clinical and educational responsibility were not evaluated, as both depend on activity the call calendar does not capture. We evaluated the system with 100 independent full-year runs (July 1 to June 30), each generated end-to-end across all call pools and months under an independently randomized vacation distribution. Compliance and fairness were evaluated at the level of the individual call-pool month (service-month), the unit at which a monthly schedule is produced; fairness and workload metrics were computed across the program’s in-program surgery residents. Results We generated 12,000 service-month schedules over 100 independent full-year runs. 99.2% (11,906) satisfied all prespecified call-related constraints with no violations detected by the internal validation engine. Forty-five of the 100 full-year runs required no manual intervention in any service-month, and none required it in more than 4 of its 120. Manual review of 120 randomly selected service-months by two authors, without reference to the validator’s output, agreed with it in 120 of 120 cases. Monthly fairness, defined within each call pool and month as the range (maximum minus minimum) of call shifts or weekends assigned per resident, showed a shift range ≤ 1 in 65.4% of service-months (mean 1.84, SD 2.15) and perfect weekend equity (range = 0) in 42.5%, with 86.6% achieving a weekend range ≤ 1 (mean 0.71, SD 0.70). Mean full-year generation time was 108.8 s (SD 59.1). Residents averaged 18.65 weekends (SD 3.13) and 78.23 call days (SD 5.49) per year. Conclusions This automated scheduling system generates resident call schedules that satisfy the prespecified call-related ACGME constraints, optimizes fairness across multiple dimensions, accommodates complex vacation requirements, handles cross-pool coordination, and flags the service-months it cannot resolve within its search budget rather than emitting a non-compliant schedule. These results provide construct validity evidence that optimization algorithms combined with careful constraint specification can eliminate the great majority of the administrative burden of manual scheduling while improving schedule quality and equity.