ChatGPT as a Digital Pharmacist: A Systematic Review and Meta-Analysis of Drug-Counselling Accuracy

Background: The emergence of Large Language Models (LLMs) like ChatGPT presents significant opportunities for healthcare, yet raises concerns about accuracy, especially in high-risk areas such as medication counseling. A comprehensive evaluation of ChatGPT's reliability in providing drug information is crucial for its safe integration into clinical practice. This systematic review and meta-analysis aimed to assess the accuracy of drug counseling information provided by ChatGPT 4. Methods: Following PRISMA, we systematically searched PubMed, Embase, Scopus, and Web of Science on May 9, 2025, for original research evaluating the accuracy of ChatGPT (version 4 or newer) in drug counseling queries. Included studies compared the AI's output against standard comparators like pharmacists or drug databases. A random-effects meta-analysis was performed to calculate the pooled proportion of accurate responses, and study quality was assessed using a customized Newcastle-Ottawa Scale (NOS). Results: The search identified 17 eligible studies. Of these, 15 were included in the meta-analysis, which showed a pooled accuracy rate of 86% (95% CI: 0.75-0.95). However, significant heterogeneity was observed across studies (I^2=98.5%, p<0.0001). The quality of the studies was a concern, with only four studies (24%) rated as high quality. No evidence of publication bias was found (p=0.91). Conclusion: ChatGPT demonstrates substantial promise in drug counseling, with an 86% accuracy rate that surpasses its performance in other medical domains. However, the high heterogeneity and a non-trivial 14% error rate, coupled with methodological weaknesses in the primary literature, indicate that ChatGPT is not yet ready for autonomous clinical use. Its current role should be as a supplementary tool under the strict supervision of qualified healthcare professionals to ensure patient safety.