Quality Assessment of Patient-Facing Urologic Telesurgery Content Using Validated Tools

Abstract Introduction: With increasing accessibility to Artificial Intelligence (AI) chatbots, the precision and clarity of medical information provided requires rigorous assessment. Urologic telesurgery represents a complex concept that patients will investigate using AI. We compared ChatGPT and Google Gemini in providing patient-facing information on urologic telesurgical procedures. Methods: 19 questions related to urologic telesurgery were generated using general information from the American Urologic Association (AUA) and European Robotic Urology Section (ERUS). Questions were organized into 4 categories (Prospective, Technical, Recovery, Other) and directly typed into ChatGPT 4o and Google Gemini 2.5 (non-paid versions). For each question, a new chat was started to prevent any continuation of answers. Three reviewers independently reviewed the responses using two validated healthcare tools: DISCERN (quality) and Patient Education Material Assessment Tool (understandability and actionability). Results: Mean DISCERN scores (out of 80) were higher for Gemini than ChatGPT in all domains except “Other”. Prospective 49.2 vs 39.1; technical 52.3 vs 44.3; recovery 53.7 vs 45.4; other 54.3 vs 56.5; overall 52.4 vs 45.8 (Figure 1). PEMAT-P understandability uniformly exceeded 70% for both platforms: prospective 80.0% vs 71.7%; technical 80.1% vs 79.8%; recovery 79.2% vs 80.1%; other 79.2% vs 81.3%; overall 79.7% vs 78.1% (Figure 2). Actionability was uniformly low; only Gemini met 70% threshold in the prospective domain (Figure 3). Conclusion: ChatGPT and Gemini deliver relevant and understandable information related to urologic telesurgery, with Gemini more consistently providing sources. However, neither chatbot reliably offers actionable responses, limiting their utility as a standalone gateway for patient decision-making.