Evaluating Postpartum Hemorrhage Transfusion Risk with a Machine Learning Model for Informed Consent: Retrospective Cohort Study.

BACKGROUND: Postpartum hemorrhage requiring a blood transfusion is a concern for patients and clinicians. Postpartum hemorrhage risk and mode of delivery are important discussions which occur before labor. Many high-risk factors associated with postpartum hemorrhage are known prior to delivery and are often unpreventable. Delivery plans are influenced by the patient's medical history, their preferences, and clinical decision making. Informed consent of known risk factors for postpartum hemorrhage will help guide delivery care plans and mitigate risk. Machine learning models have been used to predict postpartum hemorrhage, however, translation into clinical support tools is challenging. Shared decision-making discussions can be facilitated with machine learning model based clinical support tools predicting postpartum hemorrhage requiring a transfusion. OBJECTIVE: We sought to develop a machine learning model for prediction of postpartum hemorrhage requiring a transfusion. Specifically, we sought to evaluate the model's accuracy in predicting a patient's postpartum transfusion risk with delivery mode, induction of labor, and delivery indications for the purposes of antenatal clinical decision support. Model performance was evaluated on existing structured data and physician reviewed datasets. METHODS: A 10-year retrospective cohort (N = 62,521) births in a community health system were sampled. A convenience sample of (n = 1734) patients were analyzed to predict blood transfusion rate based on delivery mode and delivery indications. XGBoost, random forest, and generalized linear models were trained and compared for performance. Datasets were evaluated using the best performing XGBoost machine learning model. A prototype clinical support app for physician- patient transfusion risk assessment was developed using the best performing, clinically relevant XGBoost model. RESULTS: A generalized Linear Model, Random Forest, and XGBoost models were evaluated. The XGBoost model trained with an existing electronic medical record (EMR) extracted dataset (n=1734), performance was an AUC =.71, PR-ROC =.82, and F₁ score =.80). The dataset was then reviewed by a physician, (n=1734) performance was AUC= .705, PR-ROC =.78., and F₁ score= .809. Feature importance ranking and predictions, however, were not clinically accurate for the existing dataset. CONCLUSIONS: Machine learning models are useful to determine an individual's postpartum transfusion risk based on clinically variable and potentially modifiable factors such as delivery mode, induction of labor, and delivery indications. In this study, XGBoost model performance on EMR extracted structured data had a slightly higher AUC, (AUC =.71, PR-ROC =.82 vs AUC = .705, PR-ROC = .78), but slightly lower F₁ score (F₁ = .80 vs F₁ =.809). XGBoost machine learning models trained on clinician reviewed data can be used to predict postpartum transfusion. Clinically relevant, physician labeled datasets are important for supervised machine learning model training for use in clinical decision support tools. Further study and external validation are needed prior to clinical use.