Key result
XGBoost model using early EHR data predicts 10-day pulmonary embolism with ~0.88 AUROC.
Why the study?
Machine learning algorithms predicting pulmonary embolism onset could enable earlier treatment, but the extent to which they generalize to broader patient populations was unclear.
Does an XGBoost machine learning algorithm using early EHR data accurately predict pulmonary embolism within 10 days in hospitalized adult patients?
Observational (n=1,991,983)
Yes
Does an XGBoost machine learning algorithm using early EHR data accurately predict pulmonary embolism within 10 days in hospitalized adult patients?
Answers are grounded in this paper and cite their sources.
A machine learning model using early EHR data demonstrated excellent generalizability for predicting pulmonary embolism across diverse hospital populations.
BACKGROUND: Pulmonary embolism (PE) is a life-threatening condition associated with ~10% of deaths of hospitalized patients. Machine learning algorithms (MLAs) which predict the onset of pulmonary embolism (PE) could enable earlier treatment and improve patient outcomes. However, the extent to which they generalize to broader patient populations impacts their clinical utility. OBJECTIVE: To conduct the first large-scale external validation of a machine learning-based PE prediction model which uses EHR data from the first three hours of a patient's hospital stay to predict the occurrence of PE within the next 10 days of the inpatient stay. METHODS: This retrospective study included approximately two million adult hospital admissions across 44 medical institutions in the US from 2011 to 2017. Demographics, vital signs, and lab tests from adult inpatients at 12 institutions (n = 331,268; 3.3% PE positive) were used for training an XGBoost model. External validation of the model was conducted on patient populations from each of 32 medical institutions (total n = 1,660,715; 3.7% PE positive) without retraining. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC). Backward elimination regression was used to identify correlations between characteristics of the external validation sets and AUROC. RESULTS: The model performed well (AUROC = 0.87) on the 20% hold-out subset of the training set. Despite demographic differences between the 32 external validation populations (percent PE positive: min = 1.54%, max = 6.47%), without retraining, the model had excellent discrimination, with a mean AUROC of 0.88 (min = 0.79, max = 0.93). Fixing sensitivity at 0.80, the model had a mean specificity of 0.85 (min = 0.64, max = 0.93). Backward elimination regression identified a negative association (β = -0.015, p < 0.001) between the percentage of PE positive encounters and AUROC. CONCLUSIONS: A PE prediction model performed remarkably well across 32 different external patient populations without retraining and despite significant differences in demographic characteristics, demonstrating its generalizability and potential as a clinical decision support tool to aid PE detection and improve patient outcomes in a clinical setting.
No takes yet. Share an insight, caveat, or question.
Shen et al. (2022) conducted an observational in Pulmonary embolism (n=1,991,983). XGBoost machine learning model was evaluated on Model performance for predicting pulmonary embolism within 10 days (AUROC). An XGBoost machine learning model using early EHR data predicted pulmonary embolism within 10 days with excellent discrimination across 32 external validation populations.
Synapse has enriched 10 related papers. Consider them for comparative context: