Verification of National Weather Service Ensemble Streamflow Predictions for Water Supply Forecasting in the Colorado River Basin

The Ensemble Streamflow Prediction (ESP) system, developed by the National Weather Service (NWS), uses conceptual hydrologic models and historical data to generate a set, or ensemble, of possible streamflow scenarios conditioned on the initial states of a given basin. Using this approach, simulated historical probabilistic forecasts were generated for 14 forecast points in the Colorado River basin, and the statistical properties of the ensembles were evaluated. The median forecast traces were analyzed using “traditional” verification measures; these forecasts represented “deterministic ESP forecasts.” The minimum-error and historical traces were examined to evaluate the median forecasts and the forecast system. Distribution-oriented verification measures were used to analyze the probabilistic information contained in the entire forecast ensemble. Using a single-trace prediction, for example, the median, resulted in a loss of valuable uncertainty information about predicted seasonal volumes that is provided by the entire ensemble. The minimum-error and historical traces revealed that there are errors in the data, calibration, and models, which are part of the uncertainty provided by the probabilistic forecasts, but are not considered in the median forecast. The simulated ESP forecasts more accurately predicted future streamflow than climatology forecasts and, on average, provided useful information about the likelihood of future streamflow magnitude with a lead time of up to 7 months. Overall, the forecast provided stronger probability statements and became more reliable at shorter lead times. The distribution-oriented verification approach was shown to be applicable to ESP outlooks and appropriate for extracting detailed performance information, although interpretation of the results is complicated by inadequate sample sizes.