Large language model-generated clinical summaries in emergency departments: A blinded comparison study

Emergency department (ED) clinicians routinely construct “one-liner” summaries, distilling a patient’s history and presentation into one high-yield sentence that supports rapid decision-making. Producing these summaries is cognitively demanding. Large language models (LLMs) may assist by synthesizing longitudinal electronic health record (EHR) data. In this blinded study of 99 ED encounters, emergency physicians evaluated paired LLM- and physician-authored summaries on accuracy, completeness, and clinical utility, indicating their overall preference with free-text explanation. LLM-generated one-liner summaries were produced using k-nearest-neighbor few-shot prompting. We used linear mixed-effects to compare ratings. We also examined the LLM’s selective note inclusion and used rapid content analysis to summarize free-text explanations. We found that across all dimensions, LLM-generated summaries received higher ratings than physician-authored summaries. Mean (SE) estimated marginal means for accuracy were 4.18 (0.09) vs 3.40 (0.11) (β = 0.78; 95% CI 0.50–1.07), for completeness 3.69 (0.10) vs 3.25 (0.12) (β = 0.44; 95% CI 0.14–0.74), and for clinical utility 3.88 (0.10) vs 3.21 (0.12) (β = 0.67; 95% CI 0.35–0.99). The LLM utilized 53.2% of available notes, with History prospective validation is required before deployment in real-world ED workflows.