Key result
ChatGPT matches FDA patient leaflet quality, while DeepSeek and Gemini score significantly lower.
Why the study?
Patients increasingly use large language models for medication information, but few studies have compared AI-generated outputs with regulatory materials for oral anticoagulants.
Do AI-generated patient information leaflets for oral anticoagulants match the quality, usability, and readability of FDA-referenced materials?
Cross-Sectional (n=20)
Blinded reviewers
Do AI-generated patient information leaflets for oral anticoagulants match the quality, usability, and readability of FDA-referenced materials?
ChatGPT can generate patient information leaflets for oral anticoagulants with informational quality comparable to FDA materials, though both AI and regulatory leaflets suffer from limited actionability and high reading complexity.
Answers are grounded in this paper and cite their sources.
Patients should verify LLM anticoagulant advice against official sources; this review leaves open the need for rigorous validation studies.
INTRODUCTION: Oral anticoagulants are frequently associated with preventable hospital admissions, and patient education is essential to their safe and effective use. Patients increasingly use large language models (LLMs) for medication information, yet few studies have compared AI-generated outputs with regulatory materials or examined oral anticoagulants, a class in which misunderstanding can cause bleeding or thrombosis. AIM: To assess the informational quality, usability, readability, and output reproducibility of AI-generated patient information leaflets (PILs) for oral anticoagulants compared with FDA-referenced patient materials. METHOD: PILs for five oral anticoagulants (warfarin, apixaban, dabigatran, rivaroxaban, and edoxaban) were generated by ChatGPT, Gemini, and DeepSeek using a standardised zero-shot prompt based on FDA labelling templates; FDA-approved leaflets served as the comparator. Materials were anonymised and brand-blinded. Three clinical pharmacists independently evaluated each PIL using the Patient Education Materials Assessment Tool for Print Materials (PEMAT-P) and a modified DISCERN (mDISCERN), which assess informational quality, understandability, and actionability, but not pharmacological accuracy or clinical safety. Readability was assessed using seven validated indices. Output reproducibility was examined within the same day and at Day 1, 14, and 28. RESULTS: ChatGPT produced PILs of informational quality similar to FDA-referenced materials, both achieving a median mDISCERN score of (41/65); DeepSeek (34/65) and Gemini (33/65) scored significantly lower than ChatGPT; Gemini also scored significantly lower than the FDA-referenced materials. Understandability was acceptable for all sources, whereas actionability was limited, including in FDA materials, with no leaflet providing a patient summary or decision-support tool. All materials exceeded the recommended sixth- to eighth-grade range on most indices, with FDA leaflets the most complex. Output was stable across generations, with no significant within-model differences and the largest mean difference of 0.23 points. CONCLUSION: Among the evaluated models, ChatGPT most closely matched FDA-referenced materials in the assessed informational domains and demonstrated stable output over 28 days. Shared deficiencies across AI-generated and FDA leaflets suggest broader limitations in written health information. This comparability was limited to the assessed informational domains and does not establish equivalence in pharmacological accuracy, clinical correctness, or patient safety. AI-generated PILs may serve as clinician-reviewed supplementary materials and should not be used as standalone tools without independent professional review and verification of clinical content.
No takes yet. Share an insight, caveat, or question.
Khan et al. (2026) conducted a cross-sectional in Oral anticoagulation therapy (n=20). AI-generated patient information leaflets vs. FDA-referenced patient materials was evaluated on Informational quality (modified DISCERN score) (epsilon-squared 0.715, p=0.004). ChatGPT generated patient information leaflets with informational quality similar to FDA materials (median mDISCERN 41/65 for both), whereas DeepSeek and Gemini scored significantly lower.
Synapse has enriched 10 related papers. Consider them for comparative context: