Citation screening using large language models for creating clinical practice guidelines: A protocol for a prospective study

Abstract Background The development of clinical practice guidelines requires a meticulous literature search and screening process. This study aims to explore the potential of large language models in the development of the Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock (J-SSCG), focusing on enhancing literature search quality and reducing the citation screening workload. Methods A prospective study will be conducted to compare the efficiency and accuracy of literature citation screening between the conventional method and a novel approach using large language models. We will use the large language model, namely GPT-4, to conduct literature searches for predefined clinical questions. We will objectively measure the time required for citation screening and compare it to the time taken using the conventional method. Following the screening, we will calculate and compare the sensitivity and specificity of the results obtained from the conventional method and the large language models-assisted process. The total time spent using both approaches will also be compared to assess workload reduction. Trial registration This research is submitted with the University hospital medical information network clinical trial registry (UMIN-CTR) UMIN000053091. Conflicts of interest All authors declare no conflicts of interest to have. Funding None