Evaluation of Patient Information Provided by Artificial Intelligence Chatbots on Intravesical BCG Therapy for Bladder Cancer: A Cross-Sectional Methodological Study

Objective: Artificial intelligence (AI) chatbots are increasingly used as accessible sources of medical information. However, concerns remain regarding the quality, reliability, readability, and clinical applicability of AI-generated health content. Intravesical Bacillus Calmette-Guérin (BCG) therapy is a standard adjuvant treatment for intermediate- and high-risk non-muscle invasive bladder cancer (NMIBC) and requires adequate patient understanding of treatment procedures and potential adverse effects. This study aimed to evaluate the quality, reliability, readability, and guideline adherence of information provided by commonly used AI chatbots regarding intravesical BCG therapy. Methods: This cross-sectional methodological study evaluated responses generated by four AI chatbots (ChatGPT, Copilot, Gemini, and Perplexity) to a standardized question regarding intravesical BCG therapy. Responses were independently assessed by five urologists using validated tools including DISCERN, PEMAT-P, WRR, and the Coleman-Liau Index (CLI). Compliance with European Association of Urology (EAU) guidelines was evaluated using a 5-point Likert scale. Results: Responses ranged from 454 to 550 words and required a university-level reading ability (CLI range: 16.02–19.35). DISCERN scores indicated moderate information quality across all chatbots, with Gemini demonstrating the numerically highest median score (51 [47–59]). PEMAT-P actionability scores were generally low (median 40%). Guideline adherence was moderate, with all chatbots receiving a median Likert score of 3. Conclusion: AI chatbots provide moderate-quality information regarding intravesical BCG therapy but require high reading-grade levels and offer limited actionable guidance. AI-generated information may support patient education but should not replace physician counseling.