Answers are grounded in this paper and cite their sources.
Large language models now write clinical summaries: discharge summaries, emergency department encounter summaries, progress note summaries, referral letters and clinic visit summaries. Each summary is read, trusted and signed by a clinician, and errors in it travel into the record. This systematic review and meta-analysis asked a single question with a per-summary denominator: in studies that checked LLM-generated clinical summaries against the source record, a clinician-written summary, or blinded expert rating, what proportion of summaries contained at least one clinically significant error? Studies were searched from 2022 to 2026 across PubMed, Europe PMC (including its preprint index), Semantic Scholar and ClinicalTrials.gov, all run on 31 October 2026, with backward citation chasing of the included studies. Embase, Scopus, Web of Science, CINAHL, CENTRAL, IEEE Xplore and the ACM Digital Library were not run; OpenAlex was blocked by a provider rate limit and was not searched. Screening and extraction were each done twice independently, with every disagreement settled by reading the source. Risk of bias used a modified QUADAS-2 tool with QUADAS-AI signalling questions, specified before extraction. Proportions were pooled with a random-effects generalised linear mixed model (logit link), with tau squared, I squared and a prediction interval reported. The search found 991 records; 483 were screened after deduplication, 48 reports were assessed in full, and 11 studies (22 arms, 21 LLM and 1 human) were included. The primary outcome could not be pooled: only 1 of the 11 studies reported the proportion of summaries with at least one clinically significant error (13 of 62 summaries, 21.0 percent, exact 95 percent CI 11.7 to 33.2), and of the 48 reports assessed in full, 31 (65 percent) reported no extractable per-summary error count at all. A supplementary pool of summaries with at least one error of any audited type (post hoc, Amendment 01) gave 60.6 percent (95 percent CI 29.8 to 84.7; 5 units, 4 studies, 323 summaries; prediction interval 6.9 to 97.0). By error type, 20.7 percent of summaries had at least one omission (16 units, 1011 summaries), 17.6 percent had at least one fabrication or addition (8 units, 1226 summaries; a pool the paper warns is close to uninterpretable as a single frequency), and 27.9 percent had at least one inaccuracy (5 units, 500 summaries). One study rated 1 of 100 summaries as likely to cause moderate harm. No study reported matched LLM and human error counts, so that comparison could not be fitted. Certainty was rated very low for every rateable outcome (GRADE), and the LLM versus human comparison was not rateable. The package contains the manuscript (PDF and Markdown), the screening and extraction datasets, risk of bias assessments and the analysis datasets. Analysis code is in the linked GitHub repository. The work was carried out by an LLM research team under a single accountable human author; the team log, prompts and adjudication records are in the repository. Registration: no PROSPERO number was issued for this review. The protocol, timestamped before screening, is published inside the linked GitHub repository.
No takes yet. Share an insight, caveat, or question.
Raphael Malikian (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: