Large language models (LLMs) show promising diagnostic and triage performance, yet direct comparisons with healthcare professionals (HCPs) and collaborative effects remain limited. We conducted a systematic review and meta-analysis of studies (January 2020 to September 2025) comparing the diagnostic or triage accuracy of LLMs, HCPs, or their collaboration across seven databases. Studies using multiple-choice formats rather than open diagnostic generation were excluded. We extracted top-1, top-3, top-5, and top-10 diagnostic and triage accuracies and pooled results using multilevel random-effects models to account for nested observations. Of 10,398 studies screened, 50 met criteria, evaluating 25 different LLMs across diverse medical specialties. The relative diagnostic accuracy of LLMs versus HCPs progressively improved from 0.89 (95% CI, 0.79-1.00) for top-1 to 0.91 (0.83-1.00) for top-3, 1.04 (0.89-1.22) for top-5, and 1.17 (0.87-1.57) for top-10 diagnoses, with significant model variability. LLM-assisted HCPs outperformed HCPs alone, with relative diagnostic accuracy of 1.13 (1.00-1.27) for top-1, 1.11 (1.01-1.23) for top-3, 1.42 (1.16-1.73) for top-5, and 1.33 (0.94-1.87) for top-10 diagnoses. Triage accuracy was similar between LLMs and HCPs (1.01 0.94-1.09). These findings show potential for LLM integration but methodological flaws in studies necessitate rigorous real-world evaluation before clinical implementation.
Chen et al. (Fri,) studied this question.