From Information to Understanding: Impact of GPT-4–Generated Discharge Letters on Patients’ Medical Comprehension - A Prospective, Exploratory Study (Preprint)

BACKGROUND Patients frequently struggle to understand standard discharge letters, posing a potential risk for patient safety through medication errors and misunderstandings. AI tools that generate patient-centred versions could help bridge this gap. However, evidence for their effectiveness remains limited. OBJECTIVE This study examines the impact of GPT-4-generated patient letters on patients’ retention and understanding of safety-relevant medical information compared with conventional discharge letters. METHODS 48 trained standardized patients participated in this observatory, prospective crossover study. Each participant received one discharge letter for the assigned disease (out of three) and its matching GPT-4-generated patient letter. Participants read one version first, identified the 24 pre-defined safety-relevant “learning objectives,” then—after crossover—repeated the task with the alternate version. Primary outcome was the proportion of learning objectives fully, partially, or not reported. Secondary analyses stratified results by content field (Medication, Organization, Prevention of Complications, Lifestyle/Disease Management) and Bloom’s taxonomy level (“Remember,” “Understand”). RESULTS The letter type significantly influenced comprehension, with patient letters leading to higher rates of fully (490/1152, 42.5 % vs. 418/1152, 36.3 %) or partially stated (327/1152, 28.4 % vs. 291/1152, 25.3 %) learning objectives and fewer omissions (335/1152, 29.1 % vs. 443/1152, 38.5 %, P<.001). Participants performed better on “Remember” than “Understand” objectives, regardless of letter type (P<.001), though patient letters consistently improved results across both categories (“Remember”: 278/576, 48.3 % vs. 244/576, 42.4 %, “Understand”: 212/576, 36.8 % vs. 174//576, 30.2 % fully stated). The benefit of patient letters varied by content field (P<.01), with the greatest improvements in “Medication” (170/254, 66.9 % vs. 129/254, 50.8 % fully stated) and “Organization” (78/158, 49.4 % vs. /158, 62/158, 39.2 % fully stated). Gains in “Prevention of Complications” and “Lifestyle/Disease Management” were minimal. Nearly one-third of key information remained unrecognized across conditions. CONCLUSIONS GPT-4-generated patient letters enhanced comprehension of safety-relevant information, especially for medication and organisational details, but did not fully address higher-order understanding such as risk prevention or lifestyle change. Multimodal, interactive supports will be needed to close these residual gaps in patient education and safety.