Local Properties of k-NN Regression Estimates

Let (Xᵢ ,Yᵢ ), i = 1, ⋯ ,n be i.i.d. bivariate random vectors such that Xᵢ ∈ Rᵖ ,Yᵢ ∈ R¹. Suppose rₙ (x) denotes the k-nearest neighbor (k-NN) estimator of $r(x) = E(Y | X = x )$. Under appropriate conditions, we derive the rates of convergence for the bias and variance as well as asymptotic normality of rₙ (x). These appear to share some similarities with the k-NN density estimates. The technique is by conditioning on Rₙ, the Euclidean distance between x and its kth nearest neighbor among the Xⱼ’s. Some comparison is made between the k-NN and kernel methods.