What does this research mean for the field?

While general-purpose large language models show promise for structured mental health care applications, current evidence is too heterogeneous, indirect, and uncertain to support their routine unsupervised clinical use. Novelty: ClaimNovelty.SYNTHESIS. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

To systematically review empirical evidence evaluating the clinical usefulness, diagnostic performance, safety, and implementation readiness of general-purpose large language models in mental health care.

September 2, 2026JMIR AIOpen Access

Generative Large Language Models in Mental Health Care Settings: Systematic Review and Meta-Analysis

View Full Paper

Ask AI

Bookmark

Authors

JLJanni LeungThe University of Queensland BJBenjamin JohnsonGrand Rapids Community College KMKelsey Mcrae

Discussion

Member takes

Implication

Systematic review and meta-analysis reveals inconsistent diagnostic accuracy among large language models in mental health tasks, indicating they are not ready for unsupervised clinical use.

Key Points

To systematically review empirical evidence evaluating the clinical usefulness, diagnostic performance, safety, and implementation readiness of general-purpose large language models in mental health care.
Followed PRISMA guidelines and searched PubMed, Embase, ACM Digital Library, IEEE Xplore, and Google Scholar from November 2022 to March 2026 for quantitative evaluations of general-purpose LLMs in mental health care tasks.
Assessed study quality with the Mixed Methods Appraisal Tool and evidence certainty with GRADE domains.
Conducted random-effects meta-analyses with Hartung-Knapp adjustments and prediction intervals to pool sensitivity and specificity for GPT-4, GPT-3.5, and GPT-3 in screening and diagnosis.
Included 66 studies (37 vignette or simulation, 22 retrospective, 7 prospective) spanning screening and diagnosis (n=29), clinical decision support (n=14), treatment support (n=10), documentation and monitoring (n=6), patient education (n=4), and patient engagement (n=3).
Meta-analysis of 8 screening and diagnosis studies found pooled sensitivity was highest for GPT-4 at 0.83 (95% CI 0.38–0.97; 95% PI 0.02–1.00) compared to GPT-3.5 (0.70, 95% CI 0.13–0.97; 95% PI 0.00–1.00) and GPT-3 (0.61, 95% CI 0.33–0.82; 95% PI 0.10–0.96), while GPT-4 pooled specificity was 0.77 (95% CI 0.52–0.91; 95% PI 0.10–0.99).
Certainty of evidence was generally low across domains due to reliance on simulated vignettes, inconsistent outcome reporting, and an absence of prospective real-world clinical evaluations.

Cite This Study

Leung et al. (2026) studied this question.

synapsesocial.com/papers/6a97e2eac562ede874ec755c https://doi.org/10.2196/87730

View Full Paper

Ask AI

Bookmark

Implication

Systematic review and meta-analysis reveals inconsistent diagnostic accuracy among large language models in mental health tasks, indicating they are not ready for unsupervised clinical use.

Generative Large Language Models in Mental Health Care Settings: Systematic Review and Meta-Analysis

Discussion

Member takes

Implication

Key Points

Cite This Study

Generative Large Language Models in Mental Health Care Settings: Systematic Review and Meta-Analysis

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider