A new peer reviewer? Comparing AI with human performance in randomized controlled trial risk-of-bias assessment

Jonathan Lettner; Marko Ostojić; Aleksandra Królikowska; M. Enes Kayaalp; Nikolai Ramadanov; Robert Prill

doi:10.17219/acem/216070

April 9, 2026Advances in Clinical and Experimental MedicineOpen Access

A new peer reviewer? Comparing AI with human performance in randomized controlled trial risk-of-bias assessment

View Full Paper

Ask AI

Bookmark

Share

Key Points

The research aims to evaluate the performance of AI versus human reviewers in risk-of-bias assessments for randomized controlled trials.
Compared AI performance against human evaluators in assessing risk-of-bias.
Utilized a hybrid approach that involved pre-screening by AI followed by expert evaluation.
Analyzed the effectiveness and reliability of both methods in systematic reviews.
AI effectively automates objective components of risk-of-bias assessments.
Human reviewers excel in interpretive and context-dependent judgements.
Combining AI with expert evaluations enhances the scalability of systematic reviews without sacrificing rigor.

Ask about this paper

Answers are grounded in this paper and cite their sources.

Implication

Compares AI-assisted and human assessments of risk-of-bias in randomized trials, suggesting a hybrid approach may enhance scalability.

Abstract

AI can reliably support the automation of objective components of RoB assessment but remains limited in handling interpretive, context-dependent judgements. A hybrid approach combining AI-assisted pre-screening with expert evaluation may enhance the scalability of systematic reviews without compromising methodological rigor.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Lettner et al. (2026) studied this question.

synapsesocial.com/papers/69d9e55278050d08c1b757aa https://doi.org/10.17219/acem/216070

View Full Paper

Ask AI

Bookmark

Share

A new peer reviewer? Comparing AI with human performance in randomized controlled trial risk-of-bias assessment

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

A new peer reviewer? Comparing AI with human performance in randomized controlled trial risk-of-bias assessment

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading