Multi‐Model Ensembles in Infectious Disease and Public Health: Methods, Interpretation, and Implementation in R

Combining predictions from multiple models into an ensemble is a widely used practice across many fields with demonstrated performance benefits. Popularized through domains such as weather forecasting and climate modeling, multi‐model ensembles are becoming increasingly common in public health and biological applications. For example, multi‐model outbreak forecasting provides more accurate and reliable information about the timing and burden of infectious disease outbreaks to public health officials and medical practitioners. Yet, understanding and interpreting multi‐model ensemble results can be difficult, as there are a diversity of methods proposed in the literature with no clear consensus on which is best. Moreover, a lack of standard, easy‐to‐use software implementations impedes the generation of multi‐model ensembles in practice. To address these challenges, we provide an introduction to the statistical foundations of applied probabilistic forecasting, including the role of multi‐model ensembles. We introduce the hubEnsembles package, a flexible framework for ensembling various types of predictions using a range of methods. Finally, we present a tutorial and case‐study of ensemble methods using the hubEnsembles package on a subset of real, publicly available data from the FluSight Forecast Hub.