What question did this study set out to answer?

To evaluate the use of pre-trained language models in triaging surgical patients for anesthesia evaluations.

January 10, 2026Journal of Clinical Monitoring and Computing0 citationsOpen Access

A pre-trained language model approach for triaging surgical patients for preoperative anesthesia clinics

NXNicole Y. XuUniversity of California, San Diego OLOnkar LitakeUniversity of California, San Diego JTJeffrey TullyUC San Diego Health System

Key Result

Using pre-trained language models improved classification accuracy for triaging surgical patients, achieving an AUC of 0.70-0.74 in integrated analysis with diagnosis codes and clinical text.

Key Points

To evaluate the use of pre-trained language models in triaging surgical patients for anesthesia evaluations.
Used retrospective data from a single center's preoperative anesthesia clinic.
Assessed four language models: RoBERTa, BERT, ClinicalBERT, and PubMedBERT.
Analyzed classification for nursing phone calls versus in-person evaluations using metrics like F1-score and AUC.
Examined three data combinations: diagnosis codes, clinical text, and both.
AUC performance ranged from 0.57 to 0.74 across different analyses.
RoBERTa outperformed other models for all data types.
Combining diagnosis codes and clinical text yielded the highest AUC performance.

Structured PICO

Do pre-trained language models accurately triage surgical patients for preoperative anesthesia evaluation using clinical notes and diagnosis codes?

Population

1,761 surgical patients evaluated at a single center preoperative anesthesia care clinic, median age 60, 46.4% male.

Intervention

Pre-trained language models (RoBERTa, BERT, ClinicalBERT, PubMedBERT) using combinations of diagnosis codes and clinical text data.

Outcome

Classification of patients appropriate for a nursing preoperative phone call versus in-person clinician evaluation (assessed by AUC, F1-score, sensitivity, specificity, average precision).surrogate

Pre-trained language models using both structured diagnosis codes and unstructured clinical notes can assist in triaging surgical patients for preoperative anesthesia evaluation.

Main Result

Absolute Event Rate: 0% vs 0%

Abstract

Abstract Purpose Preoperative anesthesia evaluation is a crucial step in ensuring patient safety and optimizing perioperative care. A heterogenous patient population requiring varying levels of assessment often leads to inefficiencies and additional resource allocation. This study proposes using pre-trained language models to assist in triaging the appropriate degree of preoperative anesthesia evaluation for surgical patients. Methods Retrospective institutional data were obtained from surgical patients evaluated at a single center preoperative anesthesia care clinic. The performance of four pre-trained language models (RoBERTa, BERT, ClinicalBERT, and PubMedBERT) in the classification of which patients would be appropriate for a nursing preoperative phone call versus in-person clinician evaluation was assessed using F1-score, area under the receiver operating characteristics curve (AUC), specificity, sensitivity, and average precision. For each pre-trained language model, three different data input combinations were assessed: (1) diagnosis codes (D); (2) clinical text data (N); and (3) diagnosis codes and clinical text (D + N). The data were split into training (75%) and test set (25%). Results There were 1,761 unique patients, with an average of 12 notes per patient and a total of 46,922 clinical documents, included in the analysis. The AUC range between the four language models was highest in the D + N analyses (0.70 – 0.74), lower in the N analyses (0.58 – 0.73) and lowest in the D analyses (0.57 – 0.62). RoBERTa had the highest score compared to the other language models for all data types. Conclusions Automating integrated analysis using pre-trained language models to aid in preoperative triaging could enhance accuracy and efficiency at scale, reducing manual review and provider burden.

Ask AI

Helpful

Bookmark

View Full Paper