A Simple Nonparametric Test of Predictive Performance

This paper derives a distribution free procedure for testing the accuracy of forecasts when the focus of the analysis is on the correct prediction of the direction of change in the variable under consideration. The test applies to a general m x n contingency table and it is shown that the standard null hypothesis of independence in a contingency table implies the null hypothesis of the proposed test of predictive failure but not vice versa. As a test of predictive performance the chi-squared test of independence will, in general, be more conservative than the suggested test of predictive failure. The paper also contains two applications: A dichotomous version of the test is applied to the CBI's Industrial Trends Surveys of actual and expected price changes in the manufacturing sector, and a trichotomous version of the test is applied to the demand data from business surveys of French manufacturing industry conducted by INSEE. (This abstract was borrowed from another version of this item.)