Objectives: This study aimed to leverage a Large Language Model (LLM) to improve the efficiency and thoroughness of medical record documentation. This study focused on aiding clinical staff in creating structured summaries with the help of an LLM and assessing the quality of these AI-proposed records in comparison to those produced by doctors. Methods: This strategy involved assembling a team of specialists, including data engineers, physicians, and medical information experts, to develop guidelines for medical summaries produced by an LLM (Llama 3.1), all under the direction of policymakers at the study hospital. The LLM proposes admission, weekly summaries, and discharge notes for physicians to review and edit. A validated Physician Documentation Quality Instrument (PDQI-9) was used to compare the quality of physician-authored and LLM-generated medical records. Results: The results showed no significant difference was observed in the total PDQI-9 scores between the physician-drafted and AI-created weekly summaries and discharge notes (P = 0.129 and 0.873, respectively). However, there was a significant difference in the total PDQI-9 scores between the physician and AI admission notes (P = 0.004). Furthermore, there were significant differences in item levels between physicians’ and AI notes. After deploying the note-assisted function in our hospital, it gradually gained popularity. Conclusions: LLM shows considerable promise for enhancing the efficiency and quality of medical record summaries. For the successful integration of LLM-assisted documentation, regular quality assessments, continuous support, and training are essential. Implementing LLMs can allow clinical staff to concentrate on more valuable tasks, potentially enhancing overall healthcare delivery.
Chiang et al. (2025) studied this question.