External validation of a surgical mortality risk prediction model for inpatient noncardiac surgery in an Australian private health insurance dataset

BACKGROUND: We previously conducted a systematic review to identify surgical mortality risk prediction tools suitable for adapting in the Australian context and identified the Surgical Outcome Risk Tool (SORT) as an ideal model. The primary aim was to investigate the external validity of SORT for predicting in-hospital mortality in a large Australian private health insurance dataset. METHODS: A cohort study using a prospectively collected Australian private health insurance dataset containing over 2 million deidentified records. External validation was conducted by applying the predictive equation for SORT to the complete case analysis dataset. Model re-estimation (recalibration) was performed by logistic regression. RESULTS: The complete case analysis dataset contained 161 277 records. In-hospital mortality was 0.2% (308/161277). The mean estimated risk given by SORT was 0.2% and the median (IQR) was 0.01% (0.003%-0.08%). Discrimination was high (c-statistic 0.96) and calibration was accurate over the range 0%-10%, beyond which mortality was over-predicted but confidence intervals included or closely approached the perfect prediction line. Re-estimation of the equation did not improve over-prediction. Model diagnostics suggested the presence of outliers or highly influential values. CONCLUSION: The low perioperative mortality rate suggests the dataset was not representative of the overall Australian surgical population, primarily due to selection bias and classification bias. Our results suggest SORT may significantly under-predict 30-day mortality in this dataset. Given potential differences in perioperative mortality, private health insurance status and hospital setting should be considered as covariables when a locally validated national surgical mortality risk prediction model is developed.