Empirical Functionals and Efficient Smoothing Parameter Selection

SUMMARY A striking feature of curve estimation is that the smoothing parameter ĥ 0, which minimizes the squared error of a kernel or smoothing spline estimator, is very difficult to estimate. This is manifest both in slow rates of convergence and in high variability of standard methods such as cross-validation. We quantify this difficulty by describing nonparametric information bounds and exhibit asymptotically efficient estimators of ĥ 0 that attain the bounds. The efficient estimators are substantially less variable than cross-validation (and other current procedures) and simulations suggest that they may offer improvements at moderate sample sizes, at least in terms of minimizing the squared error. The key is a stochastic decomposition of the empirical functional ĥ 0 in terms of a smooth quadratic functional of the unknown curve. Examples include the estimation of densities, regression functions and continuous signals in Gaussian white noise.