Generative AI platforms provided dietary advice for older Chinese patients with T2DM that substantially overlapped with established guidelines (semantic similarity scores ranging from 0.747 to 0.806).
Do Generative AI platforms provide dietary advice that aligns with established guidelines for older, low-income Chinese populations with T2DM?
Generative AI platforms show substantial overlap with established diabetes dietary guidelines for older, low-income Chinese populations, but variability exists, indicating they should not be relied upon as the sole source of information.
Effect estimate: SS average scores ranging from 0.747 to 0.806
Introduction and Objective: Generative AI (Gen AI) platforms are increasingly used for health information, yet the alignment of their outputs with culturally specific diabetes dietary guidelines remains unclear. This study evaluates the extent to which AI-generated dietary guidelines align with established guidelines from the Hong Kong Health Bureau (HK) and the ADA for the older, low-income Chinese population with T2DM. Methods: Six Gen AI platforms were prompted with 10 common diabetes diet-related questions tailored to this demographic group. Reference responses were generated from HK and ADA guidelines using NotebookLM to convert guideline content into structured answers, serving as the gold standard. AI-generated responses were then compared to the guideline-based reference answers using semantic similarity (SS) analysis. Scores (between 0 and 1) were averaged across three trials per question. Results: AI-generated responses demonstrated SS average scores ranging from 0.747 to 0.806 across platforms. Conclusion: Results suggest substantial overlap with established diabetes dietary guidelines. Variability across platforms underscores the need for structured evaluation frameworks to systemically assess the alignment, reliability, and cultural relevance of Gen AI health information. Until such validated frameworks are established, Gen AI tools should not be relied upon as the sole source of dietary guidelines in this population. Disclosure E. Chiu: None. C. Young: Advisory Panel; Ended; Sanofi. Funding Touro University California College of Osteopathic Medicine, IRAP-SR Award
CHIU et al. (Fri,) conducted a other in Type 2 Diabetes. Generative AI platforms vs. Hong Kong Health Bureau and ADA guidelines was evaluated on Semantic similarity (SS) score between AI-generated responses and guideline-based reference answers (SS average scores ranging from 0.747 to 0.806). Generative AI platforms provided dietary advice for older Chinese patients with T2DM that substantially overlapped with established guidelines (semantic similarity scores ranging from 0.747 to 0.806).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: