On the Prediction of Forecast Skill

Using 10-day forecast 500 mb height data from the last 7 yr, the potential to predict the skill of numerical weather forecasts is discussed. Four possible predictor sets are described. The first, giving the consistency between adjacent forecasts, is apparently more skillful if the anomaly correlation coefficient, rather than RMS difference, is used as measure of forecast spread and forecast skill. It is concluded that much of this enhanced skill results from the dependence of the anomaly correlation coefficient on the magnitude of the forecast anomaly. It is noted that the spread between “today's” and “yesterday's” forecast is a more reliable estimate of the skill of yesterday's forecast than today's, and the implications of this on lagged-average ensemble forecasts are discussed. The impact of temporal filtering of the data in spread/skill correlations are also described. The second predictor set is derived from a regression analysis between RMS error skill scores and EOF coefficients of the forecast and/or initial 500 mb heights. The predictors themselves are large-scale anomaly patterns, some of which, towards the end of the forecast period, resemble low-frequency teleconnection patterns of the atmosphere. It is shown that forecast EOF coefficients are more skilful predictors than EOF coefficients of the initial conditions, and that when both sets of coefficients are used in the regression there is a danger of overfitting. The dependence of these patterns on the truncation of the EOF expansion and of temporal filtering is discussed. In particular, it is shown that when a severe EOF truncation is made, some of the forecast flow anomaly patterns become less geographically localized, indicating poorer predictive skill. The third predictor is defined as the RMS skill of the day-1 forecast. Both upstream and local correlations are studied. It is shown that with day-1 forecast error leading day-3 RMS error by up to 3 days, there appears to be a propagating signal, in addition to a quasi-stationary one. In general, the latter appears to be dominant. The fourth predictor is defined as the RMS difference between the forecast 500 mb height, and the initial 500 mb height. Use of this latter predictor was motivated by diagnostic studies showing relationships between interannual variability of forecast scores and interannual variability of persistence errors. These studies are partly described here. It is shown that the use of forecast persistence as a predictor gives partial skill, at least towards the end of the forecast period. The skill of the predictors are tested, and the regression coefficients derived, on data from six winters, for both regional and hemispheric skill scores. As an independent test, the predictors are also applied separately to the seventh winter period 1986/87. It is concluded that some aspects of the low-frequency component of forecast skill variability can be satisfactorily predicted, though significant high frequency variability remains unpredicted. In discussing the physical mechanisms that underlie the use of these predictors, three important components of forecast skill variability are discussed: the quality of the initial analysis, the intrinsic instability of the flow, and the role of model systematic errors. It is shown that results from the EOF predictor for the European region towards the end of the forecast period are strongly influenced by model systematic error. On the other hand, over the Pacific/North American region, growth of errors on flows with varying barotropic stability characteristics are an important component of medium-range forecast variability. This is discussed using a barotropic model with basic states defined from the results of the regression analyses for various regions. At shorter range it is suggested that growth of errors by baroclinic processes is probably dominant.