Large language models (LLMs) have shown potential in the field of emergency department (ED) triage. However, a comprehensive evaluation of their diagnostic performance and a comparison with triage nurses have not been extensively explored. To systematically evaluate the comparative triage accuracy of LLMs versus triage nurses in ED, assess their performance in identifying patients assigned to the highest-acuity triage category by the reference standard. A systematic search was conducted in CNKI, Wanfang, VIP, CBM, PubMed, Embase, Web of Science, and Cochrane Library databases from inception to March 2025. Two researchers independently screened and evaluated the quality of the selected studies and conducted meta-analyses using RevMan 5.4 and Stata 16.0. A total of 11 studies were included, involving 3088 real or simulated ED cases. The pooled odds ratio (OR) for LLM triage accuracy was 0.65 (95% CI: 0.32–1.34; I ² = 97%). For identifying patients assigned to the highest-acuity triage category by the reference standard, LLMs demonstrated a pooled sensitivity of 61% (95% CI: 48%-73%) and specificity of 97% (95% CI: 88%-99%), with statistically significant differences ( p < 0.05). The pooled area under the curve (AUC) was 0.82 (95% CI: 0.78–0.85). These estimates reflect concordance with triage category by the reference standard rather than discrimination of outcome-anchored physiological acuity. No significant publication bias was detected by Deeks’ funnel plot asymmetry test. Current LLMs demonstrate moderate discriminatory performance for identifying patients assigned to the highest-acuity triage category by the reference standard, but have limited sensitivity for these cases. LLMs may have potential as adjunctive support tools, but further prospective evaluation with patient-centered and safety-critical outcomes is required before implementation can be considered.
Cui et al. (Mon,) studied this question.