Five methodological considerations for validating LLMs in risk of bias assessment

This is the authors' abstract. We don't add key points for this paper.

I read with interest the recent article by Eisele-Metzger and colleagues, 1 which provides a timely investigation into the use of the large language model (LLM) Claude 2 for assessing the risk of bias (RoB) in randomized controlled trials.The exploration of LLMs to address the substantial resource burden of systematic reviews is of great importance. 2However, to ensure this line of inquiry yields robust and actionable insights, several methodological considerations warrant deeper discussion.I wish to raise five key critiques grounded in the broader literature on RoB reliability and LLM evaluation, which suggest opportunities to strengthen future validation studies and better define the practical role of LLMs in evidence synthesis.