A comparative study of AI chatbots and traditional medical sources for hysterectomy patient education: Assessing professionalism, readability, and patient education quality

This study compared professionalism, readability, and patient education quality between artificial intelligence (AI)-generated responses (ChatGPT and Gemini) and the American Society of Anesthesiologists (ASA) website for 8 frequently asked questions. To compare the differences in professionalism, readability, and patient education quality between AI (ChatGPT and Gemini) and the ASA website when answering 8 common hysterectomy questions, and to assess whether AI-generated content can serve as a reliable source of patient education for hysterectomy. Blinded experts evaluated professionalism, while 6 readability indices and the patient education materials assessment tool were used to assess content quality. Statistical comparisons were performed with P <.05 considered significant. ChatGPT and Gemini demonstrated significantly higher professionalism scores than the ASA website (P <.05); however, their readability was lower (P <.05). There were no significant differences in professionalism or readability between the ChatGPT and Gemini (P >.05). Although AI responses align with clinical guidelines, their low readability poses a usability concern. AI-driven content provides professional and accurate patient education on hysterectomy. However, further refinements are required to improve accessibility without compromising quality.