Evaluating performance and potential clinical benefit of the Swedish On Scene Injury Severity Prediction (OSISP) model for prehospital field triage on Norwegian trauma data

Background Rule-based field triage protocols used by Emergency Medicine Service (EMS) personnel for assessment and prioritization of trauma patients achieve suboptimal field triage performance. An On Scene Injury Severity Prediction (OSISP) model was previously developed on Swedish trauma data with indications to improve field triage performance. This study aims to apply OSISP on external Norwegian data to assess its performance and clinical impact on unseen data. Methods Adult trauma incidents involving EMS resources at the scene of incident were included. An eXtreme Gradient Boosting OSISP model was developed on Swedish trauma registry (SweTrau) data for 2013–2020 and trained to predict severely injured patients defined as new injury severity score (NISS) > 15. OSISP was evaluated on Norwegian trauma registry (NTR) data for 2017–2022. The model performance on NTR data was evaluated using overall performance measures for discrimination and calibration (receiver operating characteristic curve, precision-recall curve, area under these curves, Brier score, calibration curve, calibration slope, calibration in the large). OSISP’s clinical impact was assessed by evaluating its under- and overtriage rates and comparing to current clinical outcome and mortality. Results The raw data from SweTrau and NTR had 75,602 and 76,529 registrations, with 47,357 and 29,709 remaining after applying eligibility criteria. In the included data, there were 16.9% and 25.2% patients with NISS > 15 in SweTrau and NTR, respectively. The OSISP predictors represented age, sex, details of the patient’s condition, incident and injury details, vital signs, and administrative details. OSISP applied on NTR data yielded an overall performance of AUC ROC =0.83, AUC PR =0.64, Brier score = 0.14, calibration slope = 0.79, and calibration in the large=–0.09. On its own, OSISP reduced undertriage to 5.0% but increased overtriage to 61.0%, while the complementary use (NTR’s triage outcomes combined with OSISP’s predictions) reduced undertriage to 2.7%, overtriage to 19.7% (compare to current clinical practice with an undertriage = 56.0% and overtriage = 30.6%), and mortality from 4.1% to 3.4–3.5%. Conclusions OSISP’s overall performance was successfully validated and a potential capability to improve triage and decrease mortality was observed, also for elderly and patients from the northern region, groups often missed with current triage tools. Registration Not applicable. Trial registration Not applicable.