Development and Validation of a Model to Quantify Injury Severity in Real Time

Importance: Quantifying injury severity is integral to trauma care benchmarking, decision-making, and research, yet the most prevalent metric to quantify injury severity-Injury Severity Score (ISS)- is impractical to use in real time. Objective: To develop and validate a practical model that uses a limited number of injury patterns to quantify injury severity in real time through 3 intuitive outcomes. Design, Setting, and Participants: In this cohort study for prediction model development and validation, training, development, and internal validation cohorts comprised 223 545, 74 514, and 74 514 admission encounters, respectively, of adults (age ≥18 years) with a primary diagnosis of traumatic injury hospitalized more than 2 days (2017-2018 National Inpatient Sample). The external validation cohort comprised 3855 adults admitted to a level I trauma center who met criteria for the 2 highest of the institution's 3 trauma activation levels. Main Outcomes and Measures: Three outcomes were hospital length of stay, probability of discharge disposition to a facility, and probability of inpatient mortality. The prediction performance metric for length of stay was mean absolute error. Prediction performance metrics for discharge disposition and inpatient mortality were average precision, precision, recall, specificity, F1 score, and area under the receiver operating characteristic curve (AUROC). Calibration was evaluated using calibration plots. Shapley addictive explanations analysis and bee swarm plots facilitated model explainability analysis. Results: The Length of Stay, Disposition, Mortality (LDM) Injury Index (the model) comprised a multitask deep learning model trained, developed, and internally validated on a data set of 372 573 traumatic injury encounters (mean SD age = 68.7 19.3 years, 56.6% female). The model used 176 potential injuries to output 3 interpretable outcomes: the predicted hospital length of stay, probability of discharge to a facility, and probability of inpatient mortality. For the external validation set, the ISS predicted length of stay with mean absolute error was 4.16 (95% CI, 4.13-4.20) days. Compared with the ISS, the model had comparable external validation set discrimination performance (facility discharge AUROC: 0.67 95% CI, 0.67-0.68 vs 0.65 95% CI, 0.65-0.66; recall: 0.59 95% CI, 0.58-0.61 vs 0.59 95% CI, 0.58-0.60; specificity: 0.66 95% CI, 0.66-0.66 vs 0.62 95%CI, 0.60-0.63; mortality AUROC: 0.83 95% CI, 0.81-0.84 vs 0.82 95% CI, 0.82-0.82; recall: 0.74 95% CI, 0.72-0.77 vs 0.75 95% CI, 0.75-0.76; specificity: 0.81 95% CI, 0.81-0.81 vs 0.76 95% CI, 0.75-0.77). The model had excellent calibration for predicting facility discharge disposition, but overestimated inpatient mortality. Explainability analysis found the inputs influencing model predictions matched intuition. Conclusions and Relevance: In this cohort study using a limited number of injury patterns, the model quantified injury severity using 3 intuitive outcomes. Further study is required to evaluate the model at scale.