A Method for Producing and Evaluating Probabilistic Forecasts from Ensemble Model Integrations

The binned probability ensemble (BPE) technique is presented as a method for producing forecasts of the probability distribution of a variable using an ensemble of numerical model integrations. The ensemble forecasts are used to partition the real line into a number of bins, each of which has an equal probability of containing the “true” forecast. The method is tested for both a simple low-order dynamical system and a general circulation model (GCM) forced with observed sea surface temperatures (an ensemble of Atmospheric Model Intercomparison Project integrations). The BPE method can also be used to calculate the probability that probabilistic ensemble forecasts are consistent with the verifying observations. The method is not sensitive to the fact that the characteristics of the forecast probability distribution may change drastically for different initial condition (or boundary condition) probability distributions. For example, the method is capable of evaluating whether the variance of a set of ensemble forecasts is consistent with the verifying observed variance. Applying the method to the ensemble of boundary-forced GCM integrations demonstrates that the GCM produces probabilistic forecasts with too little variability for upper-level dynamical fields. Operational weather prediction centers including the U.K. Meteorological Office, the European Centre for Medium-Range Forecasts, and the National Centers for Environmental Prediction have been applying this method, referred to by them as Talagrand diagrams, to the verification of operational ensemble predictions. The BPE method only evaluates the consistency of ensemble predictions and observations and should be used in conjunction with additional verification tools to provide a complete assessment of a set of probabilistic forecasts.