This paper studies what we call a "robust" Diebold–Mariano type test. The unique feature of our test is that - even in the absence of any knowledge of the forecasting method - it is robust to estimation noise in the forecasts, i.e., size is kept irrespective of estimation effects induced by model fitting. We obtain this feature by a test statistic that is based on rolling-window means whose length is a vanishing fraction of the total evaluation sample. This leads to non-standard Gumbel limit laws. Other desirable features of our test are that it is easily robustified against time-varying volatility, and that it naturally uncovers time-varying differences in predictive ability under the alternative. Simulations demonstrate the benefits of our multiply robust implementation vis-à-vis several competitors. An empirical application to forecasts for several variables, horizons, vintages and methods from the Survey of Professional Forecasters illustrates the relevance of the new approach, allowing us to identify forecasters with superior models. Such conclusions are in fact impossible to infer by extant tests, since information on the models and estimation procedures behind the forecasts are typically proprietary and, hence, estimation effects cannot be factored out.
Demetrescu et al. (Thu,) studied this question.