AI‐Augmented Search for Systematic Reviews: A Comparative Analysis

Researchers are increasingly seeking to automate systematic review workflows to reduce time and labor. However, AI‐generated search strategies often contain errors that can significantly undermine the reliability of the evidence. To evaluate the relevancy and reproducibility of automated search strategies, we conducted a comparative analysis between a human‐in‐the‐loop system with a neurosymbolic AI framework (i.e., NeuroLit Navigator) and three AI systems that primarily rely on generative language models: Scite, Consensus, and Perplexity. Results showed that through an iterative human‐in‐the‐loop approach, NeuroLit Navigator produced more precise and focused search strategies aligned with domain‐specific terminologies compared to commercial LLM‐based systems, which presented issues such as lack of reproducibility and interpretability. This study highlights the potential of human‐AI collaboration in systematic review workflows, suggesting that AI should augment, rather than replace, librarian expertise. Our findings contribute to the growing field of human‐centered AI by providing a model for designing AI systems in information‐intensive domains.