A pre-trained language model approach for triaging surgical patients for preoperative anesthesia clinics

Purpose Preoperative anesthesia evaluation is a crucial step in ensuring patient safety and optimizing perioperative care. A heterogenous patient population requiring varying levels of assessment often leads to inefficiencies and additional resource allocation. This study proposes using pre-trained language models to assist in triaging the appropriate degree of preoperative anesthesia evaluation for surgical patients. Methods Retrospective institutional data were obtained from surgical patients evaluated at a single center preoperative anesthesia care clinic. The performance of four pre-trained language models (RoBERTa, BERT, ClinicalBERT, and PubMedBERT) in the classification of which patients would be appropriate for a nursing preoperative phone call versus in-person clinician evaluation was assessed using F1-score, area under the receiver operating characteristics curve (AUC), specificity, sensitivity, and average precision. For each pre-trained language model, three different data input combinations were assessed: (1) diagnosis codes (D); (2) clinical text data (N); and (3) diagnosis codes and clinical text (D + N). The data were split into training (75%) and test set (25%). Results There were 1,761 unique patients, with an average of 12 notes per patient and a total of 46,922 clinical documents, included in the analysis. The AUC range between the four language models was highest in the D + N analyses (0.70 – 0.74), lower in the N analyses (0.58 – 0.73) and lowest in the D analyses (0.57 – 0.62). RoBERTa had the highest score compared to the other language models for all data types. Conclusions Automating integrated analysis using pre-trained language models to aid in preoperative triaging could enhance accuracy and efficiency at scale, reducing manual review and provider burden.