Reliability in ensemble data assimilation

A key attribute of a probabilistic forecast system is its reliability: the degree to which forecast probabilities agree with outcome frequencies. Here, we focus on short‐lead‐time (12 h) reliability in the nonlinear background forecasts of the Ensemble of Data Assimilations (EDA) from the European Centre for Medium‐Range Weather Forecasts (ECMWF). A ‘reliability budget’, derived from consistency arguments, is used to separate the mean‐squared departures of the ensemble mean (relative to observations) into bias, ensemble variance and observation‐error contributions, along with a residual that indicates a deficiency in reliability. At these short lead times, the residual is found to be sensitive, in a local manner, to the assignment of observation errors and to the parametrization of ‘stochastic physics’, which accounts for the deficit in a model's error growth rate. In particular, the results highlight the importance of the stochastic physics parametrization to represent error growth rates fully in convective regions and suggest that current stochastic physics may be too active in subtropical anticyclones, where the mid‐tropospheric meteorology is largely characterized by time‐mean descent and radiative cooling. Other results demonstrate how the reliability budget can be used to tune observation errors, which leads to an improvement in diagnosed background reliability. Although there remains some ambiguity in the attribution of deficiencies, the budget represents a useful additional tool that can help stimulate improvements in model stochastic error representation and observation‐error estimates. Such improvements should help facilitate the development of more reliable ensemble forecasts in future.