ED-triage-agent: a multi-agent framework for human-in-the-loop emergency triage

Background Emergency Department (ED) triage requires rapid assessment of acuity under time pressure with incomplete information. Existing AI triage systems require complete clinical data, including vital signs, before generating a prediction, and so cannot support the sequential, information-gathering reality of triage in which patients accumulate in queues before vital signs are collected. Pre-vital queue prioritisation, deciding which patients are seen next before vital signs exist, remains unaddressed. Objective To design and evaluate ED-Triage-Agent (ETA), a multi-agent large language model architecture that produces a provisional, ESI-scaled acuity signal from symptom-only data for queue prioritisation, and a criterion-linked ESI recommendation once vital signs are available. Methods ETA employs five specialised agents orchestrated via LangGraph, grounded in the ESI Implementation Handbook v4 via retrieval-augmented generation. Confidence weights and decision thresholds were calibrated on 30 standardised ESI Handbook Practice Cases; performance was evaluated on the external TRIAGEAGENT benchmark using exact-match accuracy, quadratic-weighted Cohen’s κw, F1, under-triage and over-triage rates, and high-priority sensitivity, with 95% bootstrap confidence intervals. Single-model and multi-agent variants, with and without RAG, were compared against the full architecture on 30 standardised ESI Handbook Competency Cases. Results On the TRIAGEAGENT benchmark, Phase 1 pre-vital prioritisation achieved 76.39% exact-match accuracy ( κ w = 0.8787) and 95.52% high-priority sensitivity. Phase 2 post-vital classification achieved 87.04% exact-match accuracy ( κ w = 0.9090), with 0.00% significant over-triage and 0.46% significant under-triage. Misclassifications were predominantly between adjacent ESI levels (97.22% within ± 1). The chain-of-thought baseline showed 10.00% significant under-triage on the Competency Cases, versus 0.00% for ETA. Conclusion This study demonstrates the technical feasibility of a two-phase multi-agent architecture for generating provisional, ESI-scaled risk signals under partial information, with performance on standardized cases comparable to previously reported benchmark results. Clinical utility, safety, and workflow integration remain to be established through prospective validation with real ED data and clinician users.