Large Language Model–Assisted Discharge Instructions for Diabetic Kidney Disease: Comparative Evaluation Study

Background Discharge instructions are an important component of the transition from hospital to home for patients with diabetic kidney disease (DKD), whose care often involves multiple comorbidities, medications, and stage-specific management needs. However, producing complete, accurate, clear, and personalized instructions can be challenging for junior physicians. Objective This study aimed to evaluate whether large language model (LLM) assistance improves the quality of DKD discharge instructions produced by junior physicians and whether its benefits vary according to chronic kidney disease (CKD) stage. Methods The study included 101 hospitalized patients with DKD. Discharge instructions for each patient were initially drafted by a group of junior residents (resident group); subsequently, another group of residents at the same level, with the assistance of DeepSeek-V3.2 (AI-assisted group), generated a separate set of discharge instructions for the same 101 patients, resulting in a total of 202 documents. Four clinical experts, including 2 nephrologists and 2 endocrinologists, independently scored both versions of the discharge instructions using a 5-point Likert scale across 5 dimensions: completeness, accuracy, safety, clarity, and personalization. Interrater reliability was assessed using the intraclass correlation coefficient (ICC). Results Interrater reliability was excellent in both the resident group (ICC=0.817, 95% CI 0.763‐0.864; P <.001) and the AI-assisted group (ICC=0.860, 95% CI 0.816‐0.897; P <.001). Compared with resident-only instructions, AI-assisted instructions received significantly higher scores across 5 evaluation dimensions and the overall evaluation (all Bonferroni-adjusted P <.001), with large effect sizes ( r =0.741‐0.927). In the resident-only group, CKD stage was negatively correlated with accuracy, safety, clarity, and overall quality (Spearman ρ=−0.33 to −0.37; all P <.001), whereas no significant correlations were observed for most dimensions in the AI-assisted group (|ρ|<0.15; P >.05). The proportion of cases achieving a ≥1-point improvement in overall quality was 99% (100/101; 95% CI 94.6%‐100.0%). Conclusions LLM assistance improved the completeness, clarity, and personalization of DKD discharge instructions generated by junior physicians, with advantages greatest in complex cases. A human-AI collaborative model is a promising approach to reduce documentation burden.