Evaluation of model predictive ability by external validation techniques

Abstract This paper deals with the problem of evaluating the predictive ability of regression models. In some cases, model validation by internal cross‐validation technique is not enough and validation by an external test set has been suggested as an effective way of evaluating the model predictive ability. Different functions for calculating the predictive squared correlation coefficient Q 2 from an external set were proposed, which lead to occasionally different estimates of the model predictive ability and therefore to contrasting decisions about model adequacy. In this paper, advantages and drawbacks of these functions in estimating model predictive ability from some simulated datasets are discussed by comparison. Copyright © 2010 John Wiley & Sons, Ltd.