Local Minima in Cross-Validation Functions

SUMMARY The method of least squares cross-validation for choosing the bandwidth of a kernel density estimator has been the object of considerable research, through both theoretical analysis and simulation studies. The method involves the minimization of a certain function of the bandwidth. One of the less attractive features of this method, which has been observed in simulation studies but has not previously been understood theoretically, is that rather often the cross-validation function has multiple local minima. The theoretical results of this paper provide an explanation and quantification of this empirical observation, through modelling the cross-validation function as a Gaussian stochastic process. Asymptotic analysis reveals that the degree of wiggliness of the cross-validation function depends on the underlying density through a fairly simple functional, but dependence on the kernel function is much more complicated. A simulation study explores the extent to which the asymptotic analysis describes the actual situation. Our techniques may also be used to obtain other related results—e.g. to show that spurious local minima of the cross-validation function are more likely to occur at too small values of the bandwidth, rather than at too large values.