Can artificial intelligence counsel expectant mothers? Expert evaluation of ChatGPT’s educational performance in gestational diabetes mellitus

Gestational diabetes mellitus (GDM) is an increasingly common pregnancy complication associated with adverse maternal–fetal outcomes and long-term risk of type 2 diabetes. Artificial intelligence (AI) tools such as ChatGPT have emerged as potential instruments for patient education, yet their reliability and safety in obstetric counseling remain underexplored. This study aimed to evaluate the educational quality of ChatGPT-5 responses to frequently asked questions about GDM. This cross-sectional study was conducted between January and June 2025 at İzmir Tepecik Training and Research Hospital, Türkiye, following STROBE guidelines. Twenty frequently asked questions about gestational diabetes mellitus, identified from routine antenatal counseling through clinician observation and consensus, were entered into ChatGPT-5 using a standardized prompt. Thirty physicians specializing in obstetrics and gynecology from Türkiye, the United Kingdom, Canada, and Germany independently rated the AI-generated answers on a 5-point Likert scale across five domains: accuracy, comprehensiveness, clarity, safety, and appropriateness. Reliability was analyzed using Cronbach’s α and intraclass correlation coefficient (ICC). Comparative analyses were performed using Friedman and Dunn–Bonferroni tests. Overall, ChatGPT-5 achieved high ratings for accuracy (4.23 ± 0.32), clarity (4.18 ± 0.27), and appropriateness (4.09 ± 0.31), while comprehensiveness (3.92 ± 0.29) and safety (3.88 ± 0.34) scored lower. Internal consistency was strong (Cronbach’s α = 0.87) and interrater reliability was good (ICC = 0.79). Among thematic domains, “Definition, causes, and risk factors” received the highest scores (4.20 ± 0.30), whereas “Follow-up and treatment” scored the lowest (3.96 ± 0.33). Safety ratings were significantly lower than accuracy and clarity ( p = 0.018). ChatGPT-5 demonstrated high accuracy and clarity in GDM counseling, suggesting its potential use as a physician-supervised educational support tool. However, limitations in comprehensiveness and safety highlight the need for guideline-based refinement and human oversight to ensure ethical and reliable integration of AI in obstetric patient education. Not applicable. This study is an observational cross-sectional design and did not involve patient enrollment or intervention.