Development and evaluation of a clinical note summarization system using large language models

Clinical notes are a vital and detailed source of information about patient hospitalizations. However, the sheer volume and complexity of these notes make evaluation and summarization challenging. Nonetheless, summarizing clinical notes is essential for accurate and efficient clinical decision-making in patient care. Generative language models, particularly large language models such as GPT-4, offer a promising solution by creating coherent, contextually relevant text based on patterns learned from large datasets. This study describes the development of a discharge summary system using large language models. By conducting an online survey and interviews, we gather feedback from end users, including physicians and patients, to ensure the system meets their practical needs and fits their experiences. Additionally, we develop a rating system to evaluate prompt effectiveness by comparing model-generated outputs with human assessments, which serve as benchmarks to evaluate the performance of the automated model. Here we show that the model's ability to interpret diagnoses borders on humanlevel accuracy, demonstrating its potential to assist healthcare professionals in routine tasks such as generating discharge summaries. This advancement underscores the potential of large language models in clinical settings and opens up possibilities for broader applications in healthcare documentation and decision-making support. This study developed a system to support physicians in writing hospital discharge summaries. Clinical notes often include essential patient information, but their length and complexity can make it challenging to summarize them efficiently. To address this, we applied artificial intelligence (AI) techniques to help generate clear and organized summaries based on patient data. We collected input from both physicians and patients through surveys and interviews to ensure the system aligned with their needs. We also evaluated the summaries created by the system by comparing them to those written by healthcare professionals. The results showed that the AI-generated summaries were comparable in accuracy to human-written versions. This suggests that such a system could assist physicians in their documentation tasks and contribute to clearer communication during care transitions. Future applications may include other types of clinical documentation. Damasio et al. develop a discharge summary system based on large language models using feedback from both physicians and patients. The results show that the AI-generated model can accurately support the generation of summaries from clinical notes demonstrating its potential to assist healthcare professionals in routine tasks.