Does a time-series machine learning method improve forecasting of hospital discharge volume compared to simple carry-forward models in academic medical centers?
Machine learning methods can reliably forecast hospital discharge volumes up to 1 year in advance with superior absolute accuracy compared to simple carry-forward models.
Importance: Forecasting the volume of hospital discharges has important implications for resource allocation and represents an opportunity to improve patient safety at periods of elevated risk. Objective: To determine the performance of a new time-series machine learning method for forecasting hospital discharge volume compared with simpler methods. Design: A retrospective cohort study of daily hospital discharge volumes at 2 large, New England academic medical centers between January 1, 2005, and December 31, 2014 (hospital 1), or January 1, 2005, and December 31, 2010 (hospital 2), comparing time-series forecasting methods for prediction was performed. Data analysis was conducted from February 28, 2017, to August 30, 2018. Group-level data for all discharges from inpatient units were included. In addition to conventional methods, a technique originally developed for allocating data center resources, and comparison strategies for incorporating prior data and frequency of model updates, was conducted to identify the model application that optimized forecast accuracy. Main Outcomes and Measures: Model calibration as measured by R2 and, secondarily, number of days with errors greater than 1 SD of daily volume. Results: During the forecasted year, hospital 1 had 54 411 discharges (daily mean, 149) and hospital 2 had 47 456 discharges (daily mean, 130). The machine learning method was well calibrated at both sites (R2, 0.843 and 0.726, respectively) and made errors greater than 1 SD of daily volume on only 13 and 22 days, respectively, of the forecast year at the 2 sites. Last-value-carried-forward models performed somewhat less well (calibration R2, 0.781 and 0.596, respectively) with 13 and 46 errors of 1 SD or greater, respectively. More frequent retraining and training sets of longer than 1 year had minimal effects on the machine learning method's performance. Conclusions and Relevance: Volume of hospital discharges can perhaps be reliably forecasted using simple carry-forward models as well as methods drawn from machine learning. The benefit of the latter does not appear to be dependent on extensive training data and may enable forecasts up to 1 year in advance with superior absolute accuracy to carry-forward models.
McCoy et al. (Fri,) studied this question.