A Prompt-Based Tutorial for Large Language Model–Assisted Screening in Systematic Reviews and Meta-Analyses

Objective: Title/abstract and full-text screening are among the most time-consuming stages of systematic reviews.Large language models (LLMs) such as ChatGPT can assist in screening, but most prior evaluations focus only on abstract-level decisions and require coding or API access, limiting practical application.This study aimed to develop and evaluate a structured, promptbased approach that enables LLMs to perform both abstract and full-text screening without programming or API integration.Methods: Using datasets from two completed meta-analyses, we implemented a stepwise training framework involving comprehension checks, criterion-specific feedback, and iterative prompt refinement.Study 1 optimized technical parameters, including file format and batch size, using 1,000 abstracts and 50 full texts from a completed meta-analysis.Study 2 and 3 validated the approach using independent datasets from two meta-analyses focusing on the efficacy of teletherapy versus in-person therapy (1,321 abstracts, 82 full texts) as well as emotion regulation and mental health (1,500 abstracts, 129 full texts) respectively.Human reviewers' decisions served as the reference standard, and sensitivity, specificity, and accuracy were the primary outcomes. Results:In Study 1, the LLM achieved 98.0 percent sensitivity and 80.6 percent specificity at abstract screening, with optimal performance using batches of 50 abstracts in plain-text format.In Study 2, the LLM reached high sensitivity (100 percent) and specificity (85.6 percent) for abstracts, and 82.1 percent accuracy for full-text screening, correctly retaining all eligible studies.In Study 3, we achieved this similar level of specificity and sensitivity, with 100.0 percent sensitivity and 93.0 percent specificity at abstract screening, and 96.2 percent sensitivity and 90.3 percent specificity at the full-text stage.