Abstract Purpose The volume of residency applications and data per applicant are increasing with emphasis on holistic review and application inflation. Studies have shown artificial intelligence (AI) could augment human review in resident selection and reveal successful candidates who may otherwise be overlooked. This study determines whether AI successfully predicts match outcomes for ophthalmology residents and could be a valid means of improving the objectivity and efficiency of the residency match process. Method This was a prospective study of 642 applicants in the 2023-2024 San Francisco Match cycle. A total of 129 US doctors of ophthalmology and foreign medical graduates were excluded from analysis. The application data of the 513 US medical doctor graduates was studied to predict their match outcomes. Data were received from applicants on September 1, 2023, and were prospectively analyzed by both AI and faculty until the match results were released on February 6, 2024. Faculty utilized a standardized rubric to generate a rank list. In late September 2024, GPT 3.5 Turbo (Azure OpenAI) was given 5 main criteria and no prior examples to prevent bias. Both faculty and AI had access to the entire San Francisco Match application. Main outcome was predictiveness of rank list on match outcome of each applicant. Results Both the AI rank list and faculty rank list were predictive of matching to an ophthalmology residency spot (P values .001). Each 10-percentile increase of the AI ranking had a 23% increase in the odds of a match (odds ratio = 1.23; 95% CI, 1.15-1.32), and each 10-percentile increase of the faculty rank list had a 41% increase in the odds of a match (odds ratio = 1.41; 95% CI, 1.29-1.53). Conclusions AI accurately predicts match outcomes and can be used as an adjunct aide to faculty review of applications to reduce the immense administrative workload and human bias. Teaser Text The residency application process is growing increasingly complex as applicant volume and emphasis on holistic review continue to rise, placing significant strain on faculty reviewers. Artificial intelligence (AI) has the potential to augment human judgment by improving efficiency and objectivity in resident selection. In this prospective study of over 500 U.S. allopathic applicants in the 2023–2024 ophthalmology SF Match cycle, we evaluated whether an AI model could predict match outcomes using complete application data. AI-generated rankings were prospectively compared with faculty rankings created using a standardized rubric. Both approaches were significantly predictive of match success. These findings demonstrate that AI can meaningfully support residency application review, offering a scalable adjunct to faculty assessment that may reduce administrative burden and help mitigate human bias while preserving faculty oversight.
Moon et al. (Wed,) studied this question.