Large-Scale Artificial Intelligence or Medical Residency: Who Agrees More With Infectious Disease Specialists in Antibiogram-Guided Antibiotic Selection?

Optimizing antibiotic therapy is essential given the growing antimicrobial resistance. Variability in therapeutic decision-making among health professionals is a challenge, especially across different stages of training. In this context, the rise of large language models (LLMs) in healthcare raises questions about their clinical accuracy. This study compared the agreement of medical residents and three LLMs with infectious disease specialists in antibiotic selection. This cross-sectional study evaluated 87 cases, each combining a real urine culture antibiogram with a simulated clinical scenario (severity, pregnancy/lactation, oral route feasibility, infection site). Decisions were made blindly and independently by a first-year resident (R1), a second-year resident (R2), three LLMs (ChatGPT-4.1, Gemini 2.5 Pro, and DeepSeek V3-0324), and two infectious disease specialists (E1 and E2), who served as the reference. We calculated the agreement proportions of each participant with at least one of the two specialists (primary outcome). Statistical significance of differences was assessed using the chi-square test (χ²), with p<0.05. For the primary outcome, agreement proportions were: R1 = 47.13% and R2 = 64.37%. In comparison, the LLMs showed: Gemini = 64.37%, ChatGPT = 58.62%, and DeepSeek = 49.43%. In direct comparisons for the primary outcome, Gemini and R1 had similar agreement (p=1.000), as did DeepSeek and R2 (p=0.762), and ChatGPT and R1 (p=0.436). However, Gemini showed higher agreement than R2 (p=0.022), and DeepSeek showed higher agreement than R1 (p=0.047). There was no significant difference between ChatGPT and R2 (p=0.12884). Generic LLMs, even without medical training, matched or outperformed the first-year resident and approached the second-year resident’s performance in antibiogram-guided antibiotic selection. These findings suggest that decisions based on objective data can be automated with agreement comparable to clinicians in training. This highlights the need to (i) strengthen resident education in interpreting microbiological tests; (ii) evaluate LLMs as interactive tutors or second opinions; and (iii) develop continuous validation protocols. AI tools do not replace clinical judgment, but may raise the minimum quality threshold when integrated into supervised prescribing workflows.