Predictive Modeling to Identify Children With Complex Health Needs At Risk for Hospitalization

BACKGROUND: Identifying children at high risk with complex health needs (CCHN) who have intersecting medical and social needs is challenging. This study's objectives were to (1) develop and evaluate an electronic health record (EHR)-based clinical predictive model ("model") for identifying high-risk CCHN and (2) compare the model's performance as a clinical decision support (CDS) to other CDS tools available for identifying high-risk CCHN. METHODS: This retrospective cohort study included children aged 0 to 20 years with established care within a single health system. The model development/validation cohort included 33 months (January 1, 2016-September 30, 2018) and the testing cohort included 18 months (October 1, 2018-March 31, 2020) of EHR data. Machine learning methods generated a model that predicted probability (0%-100%) for hospitalization within 6 months. Model performance measures included sensitivity, positive predictive value, area under receiver-operator curve, and area under precision-recall curve. Three CDS rules for identifying high-risk CCHN were compared: (1) hospitalization probability ≥10% (model-predicted); (2) complex chronic disease classification (using Pediatric Medical Complexity Algorithm PMCA); and (3) previous high hospital utilization. RESULTS: Model development and testing cohorts included 116 799 and 27 087 patients, respectively. The model demonstrated area under receiver-operator curve = 0.79 and area under precision-recall curve = 0.13. PMCA had the highest sensitivity (52.4%) and classified the most children as high risk (17.3%). Positive predictive value of the model-based CDS rule (19%) was higher than CDS based on the PMCA (1.9%) and previous hospital utilization (15%). CONCLUSIONS: A novel EHR-based predictive model was developed and validated as a population-level CDS tool for identifying CCHN at high risk for future hospitalization.