What does this research mean for the field?

Large language models generate generally favorable patient information on bacteriophage therapy, but require clinical expert refinement due to a 20% rate of incorrect information. Novelty: ClaimNovelty.INCREMENTAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

To evaluate the quality, accuracy, completeness, clarity, and tone of large language model responses to patient inquiries regarding bacteriophage therapy.

September 2, 2026npj VirusesOpen Access

Performance of large language models as a source of clinical information on bacteriophage therapy

View Full Paper

Ask AI

Bookmark

Authors

NWNike Walter DADerek F. Amanatullah LDLaurent Debarbieux

Discussion

Member takes

Overview

Expert evaluation study uncovers notable factual inaccuracies in AI-generated responses to patient questions, highlighting the need for clinician oversight before clinical use.

Key Points

To evaluate the quality, accuracy, completeness, clarity, and tone of large language model responses to patient inquiries regarding bacteriophage therapy.
Twelve bacteriophage therapy experts independently assessed LLM responses to 20 patient-relevant questions across accuracy, completeness, clarity, and tone/empathy using 5-point Likert scales (N=960 total ratings).
Adjusted mean scores ranged from 3.36 to 3.96 across domains, with significant differences among LLMs in completeness and tone/empathy (Holm-adjusted p = 0.042 for both; Cohen’s d = 0.12–0.29), but not in accuracy or clarity.
Claude achieved significantly lower completeness and tone/empathy scores, while Perplexity scored highest in completeness.
Experts recommended revisions for 34% to 40% of model responses and identified incorrect clinical information in 20% of answers.

Cite This Study

Walter et al. (2026) studied this question.

synapsesocial.com/papers/6a97e318c562ede874ec7b4c https://doi.org/10.1038/s44298-026-00224-2

View Full Paper

Ask AI

Bookmark

September 2, 2026npj VirusesOpen Access

Performance of large language models as a source of clinical information on bacteriophage therapy

View Full Paper

Ask AI

Bookmark

Authors

NWNike Walter DADerek F. Amanatullah LDLaurent Debarbieux

Discussion

Member takes

Overview

Expert evaluation study uncovers notable factual inaccuracies in AI-generated responses to patient questions, highlighting the need for clinician oversight before clinical use.

Key Points

To evaluate the quality, accuracy, completeness, clarity, and tone of large language model responses to patient inquiries regarding bacteriophage therapy.
Twelve bacteriophage therapy experts independently assessed LLM responses to 20 patient-relevant questions across accuracy, completeness, clarity, and tone/empathy using 5-point Likert scales (N=960 total ratings).
Adjusted mean scores ranged from 3.36 to 3.96 across domains, with significant differences among LLMs in completeness and tone/empathy (Holm-adjusted p = 0.042 for both; Cohen’s d = 0.12–0.29), but not in accuracy or clarity.
Claude achieved significantly lower completeness and tone/empathy scores, while Perplexity scored highest in completeness.
Experts recommended revisions for 34% to 40% of model responses and identified incorrect clinical information in 20% of answers.

Cite This Study

Walter et al. (2026) studied this question.

synapsesocial.com/papers/6a97e318c562ede874ec7b4c https://doi.org/10.1038/s44298-026-00224-2

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

View Full Paper

Ask AI

Bookmark