Mp23-01 Aua Guideline Committee Members Determine Quality of Chatgpt Generated Responses for Female Stress Urinary Incontinence

You have accessJournal of UrologyUrodynamics/Lower Urinary Tract Dysfunction/Female Pelvic Medicine: Female Incontinence (MP23)1 May 2024MP23-01 AUA GUIDELINE COMMITTEE MEMBERS DETERMINE QUALITY OF CHATGPT GENERATED RESPONSES FOR FEMALE STRESS URINARY INCONTINENCE Annie Chen, Kuemin Hwang, Jerril Jacob, Marshall Fettig, Tarek Dawamne, Kathleen Kobashi, and Ricardo R. Gonzalez Annie ChenAnnie Chen , Kuemin HwangKuemin Hwang , Jerril JacobJerril Jacob , Marshall FettigMarshall Fettig , Tarek DawamneTarek Dawamne , Kathleen KobashiKathleen Kobashi , and Ricardo R. GonzalezRicardo R. Gonzalez View All Author Informationhttps://doi.org/10.1097/01.JU.0001008776.99097.8a.01AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Stress urinary incontinence (SUI) affects many women worldwide. ChatGPT, an artificial intelligence language model, had over 180 million users in 2023. Given its rising ubiquity, healthcare consumers may turn to the platform for SUI advice. Our objective was to evaluate the quality of information available to patients about SUI from the ChatGPT platform. METHODS: A search was performed for the most commonly asked questions regarding SUI derived from patient materials available through SUFU, AUA, and patient forums. Questions were subdivided by type (Definition, Diagnosis, Management, and Surgery-specific) and queried using ChatGPT. A survey was delivered to 3 AUA guideline committee members who developed the Surgical Management of Female SUI guidelines. They rated the responses on reliability, understandability, overall quality, and actionability using the DISCERN and PEMAT standardized questionnaires. Accuracy was assessed with a 4-point Likert scale (1=not accurate, 4=always accurate). Each generated response was evaluated for readability using the Flesch Reading Ease score. RESULTS: The material was rated as moderate to moderately-high quality (DISCERN=3.73/5) with potentially important but no serious shortcomings. Reliability and quality were reported to be 63% and 75% respectively. Understandability was rated 89% and actionability 18%. Overall accuracy was 88%. All domains were rated at moderate or better. Actionability was poor in all domains. Readability metrics were found to be "hard to read" for every response (average=24.9; scale 0=very difficult to read, 100=very easy to read), which translates to a reading level of a 20 year-old. CONCLUSIONS: Given ChatGPT's recent controversy regarding fabrication of information, it's important for the urologic community to critically evaluate this platform's output if patients are to use it for adjunctive medical guidance. AUA Committee members, who comprise the premier experts in the field, rate ChatGPT-produced responses on SUI as moderate to moderately high quality, moderate reliability, excellent understandability, poor actionability utilizing standardized questionnaires. The reading level of the material was advanced and suited for a typical person over the age of 20, which is an area of potential improvement. Source of Funding: None © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e381 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Annie Chen More articles by this author Kuemin Hwang More articles by this author Jerril Jacob More articles by this author Marshall Fettig More articles by this author Tarek Dawamne More articles by this author Kathleen Kobashi More articles by this author Ricardo R. Gonzalez More articles by this author Expand All Advertisement PDF downloadLoading ...