Consistent Nonparametric Regression

Let $(X, Y)$ be a pair of random variables such that X is Rᵈ-valued and Y is Rd'-valued. Given a random sample (X₁, Y₁), ⋯, (Xₙ, Yₙ) from the distribution of $(X, Y)$, the conditional distribution PY(• X) of Y given X can be estimated nonparametrically by P̂ₙY(A X) = ∑ⁿ₁ Wₙᵢ(X)IA(Yᵢ), where the weight function Wₙ is of the form Wₙᵢ(X) = Wₙᵢ(X, X₁, ⋯, Xₙ), 1 i n. The weight function Wₙ is called a probability weight function if it is nonnegative and ∑ⁿ₁ Wₙᵢ(X) = 1. Associated with P̂ₙY(• X) in a natural way are nonparametric estimators of conditional expectations, variances, covariances, standard deviations, correlations and quantiles and nonparametric approximate Bayes rules in prediction and multiple classification problems. Consistency of a sequence ₙ\ of weight functions is defined and sufficient conditions for consistency are obtained. When applied to sequences of probability weight functions, these conditions are both necessary and sufficient. Consistent sequences of probability weight functions defined in terms of nearest neighbors are constructed. The results are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.