Developing predictive models for COVID-19 positive tests based on the XGBoost and random forest algorithms with internet search data

Abstract Background Although strategies for COVID-19 have shifted towards normalized measures globally, establishing predictive models based on Internet search data remains crucial for swiftly controlling and preventing future outbreaks. This study aims to utilize Internet search data for early epidemic surveillance and warning. Methods We collected the daily number of COVID-19 positive tests and the daily Baidu Search Index (BSI) of COVID-19 related keywords. First, we screened keywords with a maximum correlation coefficient exceeding 0.9 by time-lagged correlation analysis. Then, we used the original and lagged BSI to construct XGBoost and Random Forest (RF) models for short-term prediction of the COVID-19, respectively. Next, we selected top 5 important predictors according to the importance gain in XGBoost model and constructed a comprehensive search index (CSI) weighted by the importance gain. Finally, we used the distributed lagged nonlinear model (DLNM) to evaluate the relationship between the CSI and the number of COVID-19 positive tests. Results We identified 20 keywords had a maximum correlation coefficient exceeding 0.9 with lag days of 1–10 days. Then, we found that the predictive performance of the XGBoost models was better than that of the RF models. And XGBoost models using lagged BSI (compared to original BSI) had a better predictive performance for forecasting 3 days, with an RMSE of 803.85 and a MAPE of 9.96%. Finally, we observed that the CSI was statistically associated with the number of COVID-19 positive tests, with the maximum relative risks (RR) at lags of 0, 3, 5, and 7 days being 2.18 (95% CI 1.60–2.97), 1.94 (95% CI 1.10–3.43), 1.86 (95% CI 1.01–3.44), and 2.03 (95% CI 1.00-4.11), respectively. Conclusions The XGBoost model with the lagged BSI can predict COVID-19 epidemics, which make it a powerful addition to the traditional surveillance systems.