Key result
Large language models accurately identify preoperative frailty from clinical notes with AUCs up to 0.99.
Why the study?
Frailty increases postoperative morbidity and mortality, but identifying it from medical records is challenging because it is a multifactorial state accumulated across multiple organ systems over time.
Can large language models accurately identify preoperative frailty from clinical notes in older adults?
Can large language models accurately identify preoperative frailty from clinical notes in older adults?
Large language models can accurately identify preoperative frailty from unstructured clinical notes, offering a potential automated screening tool.
May support automated frailty screening from notes; leaves open prospective validation before clinical adoption.
BACKGROUND: Patients with frailty have a higher risk of major postoperative mortality and morbidity. Identifying frailty from the medical record, however, is not straightforward since it is a multifactorial state based on multiple organ systems and a sum of factors accumulated over time. The objective of this study was to develop a large language model-based binary classifier using accurately phenotyped datasets to identify preoperative frailty from clinical notes. METHODS: We trained various large language models to identify frailty from anesthesia preoperative clinic notes. There were two development datasets used: (1) patients undergoing spine surgery whose frailty was characterized by patient responses to the Vulnerable Elders-13 Survey (VES-13); and (2) patients undergoing surgery whose frailty was characterized by their calculated electronic frailty index (eFI) score. RESULTS: When trained on our VES-13 development set and tested on our VES-13 validation set, the area under the receiver operating characteristics curve (AUC) for the RoBERTa, BERT, BioBERT, and PubMedBERT models was 0.99, 0.64, 0.67, and 0.73, respectively. When tested on the eFI validation set, the AUCs were 0.63, 0.83, 0.87, and 0.87, respectively. Models trained on the eFI development dataset did not discriminate frailty adequately when tested on the VES-13 validation set. CONCLUSION: We report the development and validation of a classifier that detects older adults at risk for preoperative frailty from preoperative anesthesia clinical notes. Large language models can be used to accurately identify a difficult-to-quantify and multifactorial characteristic such as frailty in patients by using readily available unstructured information from clinical notes.
No takes yet. Share an insight, caveat, or question.
Zhou et al. (2025) studied Preoperative frailty. Large language models was evaluated on Area under the receiver operating characteristics curve (AUC) for identifying frailty. Large language models can accurately identify preoperative frailty from clinical notes, achieving AUCs up to 0.99 for RoBERTa on VES-13 and 0.87 for PubMedBERT on eFI datasets.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: