Interpretable Machine Learning Models for Predicting Critical Outcomes in Patients with Suspected Urinary Tract Infection with Positive Urine Culture

(1) Background: Urinary tract infection (UTI) is a leading cause of emergency department visits and hospital admissions. Despite many studies identifying UTI-related risk factors for bacteremia or sepsis, a significant gap remains in developing predictive models for in-hospital mortality or the necessity for emergent intensive care unit admission in the emergency department. This study aimed to construct interpretable machine learning models capable of identifying patients at high risk for critical outcomes. (2) Methods: This was a retrospective study of adult patients with urinary tract infection (UTI), extracted from the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database. The critical outcome is defined as either in-hospital mortality or transfer to an intensive care unit within 12 h. ED visits were randomly partitioned into a 70%/30% split for training and validation. The extreme gradient boosting (XGBoost), random forest (RF), and support vector machine (SVM) algorithms were constructed using variables selected from the stepwise logistic regression model. The XGBoost model was then compared to the traditional model and clinical decision rules (CDRs) on the validation data using the area under the curve (AUC). (3) Results: There were 3622 visits among 3235 unique patients diagnosed with UTI. Of the 2535 patients in the training group, 836 (33%) experienced critical outcomes, and of the 1087 patients in the validation group, 358 (32.9%) did. The AUCs for different machine learning models were as follows: XGBoost, 0.833; RF, 0.814; and SVM, 0.799. The XGBoost model performed better than others. (4) Conclusions: Machine learning models outperformed existing traditional CDRs for predicting critical outcomes of ED patients with UTI. Future research should prospectively evaluate the effectiveness of this approach and integrate it into clinical practice.