Answers are grounded in this paper and cite their sources.
Pilot study demonstrates improved response quality for urogynecology queries with retrieval-augmented AI, suggesting advances in patient-centered care.
Introduction and Hypothesis The objective was to develop a retrieval-augmented ChatGPT model grounded in evidence-based patient education materials and compare its performance against the standard ChatGPT model in responding to common urogynecology patient questions in this pilot study. Methods We developed a retrieval-augmented ChatGPT-4.0 model that prioritized content from International Urogynecological Association patient information leaflets. Ten commonly asked patient questions were submitted to both the standard and retrieval-augmented models. Six board-certified urogynecologists evaluated responses using the validated Quality Analysis of Medical Artificial Intelligence (QAMAI) tool, which assesses accuracy, clarity, relevance, completeness, usefulness, and sources. Total and domain-specific QAMAI scores were compared using the Wilcoxon signed-rank test, and a sensitivity analysis was performed, excluding the unblinded Source domain. Results The retrieval-augmented model demonstrated significantly higher total QAMAI scores than the standard model (median 22 [interquartile range, IQR, 19–25] vs 16 [IQR 13–18], p < 0.01) and outperformed the standard model in all six domains. In the sensitivity analysis, the retrieval-augmented model maintained significantly higher performance (18 [IQR 16–20] vs 14.5 [IQR 11–17], p < 0.01). Clinician raters preferred the retrieval-augmented model in 81% of responses. Conclusions Grounding AI tools in vetted patient education materials significantly improved the quality of ChatGPT-generated responses in urogynecology. Retrieval-augmented models offer a promising approach to enhance patient education and promote patient-centered care.
No takes yet. Share an insight, caveat, or question.
Moureau et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: