Enhancing Patient Education on the Heart-Healthy Diet With Generative Artificial Intelligence Models

Objectives: Profile the readability of heart-healthy diet patient education created by generative artificial intelligence (AI) models. Methods: This study extracted the 20 most frequently asked questions regarding the heart-healthy diet using the Google ® RankBrain algorithm. Questions were extracted and categorized as "Fact", "Policy", or "Value" questions using Rothwell's Classification of Questions. Copilot (Microsoft ®) and Generative-Pre-Trained Transformer-3.5 (ChatGPT) (OpenAI ®) were employed as generative AI models to respond to each question. For the control, the associated educational articles provided with each question were employed. Readability was scored by a healthcare provider (e.g., Flesch Reading Ease, Flesch-Kincaid) for the educational articles and responses by the generative AI models. Results: The mean Flesch Reading Ease score was highest in the control (60.2 ± 8.7) followed by the Copilot (50.3 ± 8.8) and ChatGPT models (41.8 ± 10.6). One-way analysis of variance (ANOVA) demonstrated a statistically significant difference in Flesch Reading Ease between groups (p= 0.004). The mean Flesch-Kincaid score was lowest in the control (7.9 ± 1.7) followed by Copilot (8.8 ± 1.4) and ChatGPT (10.8 ± 1.9) models and demonstrated statistical significance using ANOVA (p= 0.019). Among Rothwell's Classification of Questions, a majority of questions were classified as Fact-based (n= 13) followed by Value-based (n= 6) and Policy-based (n= 1). Conclusions: This study demonstrated there is a statistically significant difference in the readability quality of generative AI models and publicly available educational articles on the heart-healthy diet. ChatGPT performed inferior in both readability metrics. Moreover, the mean Flesch-Kincaid of the ChatGPT model would not meet U.S. grade-reading level recommendations. This could suggest greater difficulty for patients in understanding heart-healthy diet education. Funding Sources: No funding was used in this study. AI Disclosures: No AI-assisted technologies were used in the writing of this study. However, the author(s) used ChatGPT and Microsoft Copilot as independent variables given the primary objective of this study.