Randomized Controlled Trial Comparing AI-Generated, Clinician-Edited to Human-Generated After-Visit Summaries

Objective To test whether AI-generated, clinician-edited after-visit summaries (AVS) are more patient-friendly than clinician-generated AVS, without compromising safety, when implemented in a live inpatient setting. Patients and Methods Patients being discharged from inpatient medicine units between January 22, 2025 and May 23, 2025 were randomized to the intervention arm, which included AI-generated, clinician-edited AVS, or the control arm, which included usual care of clinician-only AVS. Outcomes include text between 6 th and 8 th grade reading levels, and presence of a "simplified" description of both the diagnosis and treatment/interventions. Our composite outcome was positive if all three of these outcomes were positive. We also surveyed patients, nurses, and clinicians on perceptions of safety, empathy, and understandability. Results Among 61 patients enrolled in the trial, the AI-generated, clinician-edited AVS were significantly more likely to have a mean reading level closer to 6 th – 8 th grade than the control group (8.5 vs 10.6, p < 0.001). The AI-generated, clinician-edited AVS were also more likely to contain simplified explanation of diagnoses (96.7% versus 12.9%, p < 0.001), and hospital interventions (86.7% versus 12.9%, p < 0.001). Our composite outcome favored AI-generated, clinician-edited summaries (13.3% vs 6.5%, p=0.637). Conclusion In this small, randomized trial, exploratory and component measures suggest improved patient-friendliness in AI-generated, clinician-edited AVS, although no significant difference was observed in the primary outcome.