Model validation by permutation tests: Applications to variable selection

Regression model validation by permutation tests was explored. Especially in cases where the model significance is doubtful, a permutation test adds crucial information which can often can be decisive for the existence of the model. The background and applicability of the test procedure are described. As an example, the use of permutation tests was extended to validation and investigation of four predictor variable selection techniques, namely MUSEUM, GOLPE, VIP and IVS-PLS. The selection methods are briefly reviewed and compared. The permutation tests were applied before, during and after variable selection. Some similarities and differences in the behaviour of the variable selection techniques were found and are commented upon. © 1996 by John Wiley & Sons, Ltd.