The Kernel Estimate of a Regression Function in Likelihood-Based Models

Abstract Smoothing splines have a penalized likelihood motivation (Good and Gaskins 1971) allowing direct application to nonparametric regression in likelihood-based models. The notion of a weighted likelihood for the nonparametric kernel estimation of a regression function is proposed, generalizing the local likelihood theory of Tibshirani and Hastie (1987). Let the data be of the form (xi, Yi ) (i = 1, …, n), where xi ∈ 0, 1 d are lattice points and the Yi are independent random variables from a family of distributions with parameter λi = g(xi ), with g having continuous partial derivatives of order k ≥ 2. The goal is to arrive at a nonparametric estimate λ o of λ o = g(xo ) for a fixed point xo ∈ 0, 1 d. We consider the estimator λ o that maximizes the weighted likelihood function W(λ) = Σ n i=1 W(xo – xi )/b log f(Yi :λ), with f the density of Yi, W a symmetric kernel with compact support, and b the bandwidth that controls the degree of smoothing. Sufficient conditions for consistency and asymptotic normality of λ o are given. If the Yi are normal random variables with mean λi and equal variance, then λ o is the kernel estimator of Priestly—Chao (1972). It is a weighted average of Yi corresponding to xi in a neighborhood of xo . The kernel governs the weights and the bandwidth controls the size of the neighborhood. The kernel estimator of the relative risk function is developed for censored survival times under the assumption of the Cox proportional hazards model. The weighted likelihood approach based on the full likelihood is illustrated with real and simulated data.