What does this research mean for the field?

Formula-constrained prompt engineering significantly improves the accuracy and expert-alignment of large language models when assessing the risk of bias in randomized controlled trials. Novelty: ClaimNovelty.INCREMENTAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

The study aims to assess how prompt engineering influences the performance of large language models in risk of bias assessments.

May 19, 2026BMJ evidence-based medicine

Impact of prompt engineering on large language models for risk of bias assessment: a comparative study

View Full Paper

Ask AI

Bookmark

Authors

YXYu-Tao XiongSichuan University YWYuan-Chen WangSichuan University YSYuanhao SunDalian Ocean University

Discussion

Member takes

Implication

Randomized trial evaluates prompt engineering's effect on large language models for risk of bias assessment, suggesting improved accuracy and alignment with expert reasoning.

Key Points

The study aims to assess how prompt engineering influences the performance of large language models in risk of bias assessments.
Evaluated 158 randomized controlled trials from 10 dental systematic reviews.
Compared two large language models, DeepSeek-V3 and GPT-5, across four prompting strategies.
Measured accuracy and agreement metrics according to RoB-1 domains.
Mean accuracy under blank control was 0.72 for DeepSeek-V3 and 0.65 for GPT-5.
With formula-constrained prompting, accuracy rose to 0.85 for both models (p<0.001).
Formula-constrained prompts resulted in significantly higher agreement and better alignment with expert reasoning (MMD² results).

Cite This Study

Xiong et al. (2026) studied this question.

synapsesocial.com/papers/6a0bfd7a166b51b53d378dda https://doi.org/10.1136/bmjebm-2025-114055

View Full Paper

Ask AI

Bookmark

Implication

Randomized trial evaluates prompt engineering's effect on large language models for risk of bias assessment, suggesting improved accuracy and alignment with expert reasoning.

Impact of prompt engineering on large language models for risk of bias assessment: a comparative study

Discussion

Member takes

Implication

Key Points

Cite This Study

Impact of prompt engineering on large language models for risk of bias assessment: a comparative study

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider