On the Almost Everywhere Convergence of Nonparametric Regression Function Estimates

Let (X, Y), (X₁, Y₁), , (Xₙ, Yₙ) be independent identically distributed random vectors from Rᵈ R, and let E (|Y|ᵖ) < for some p 1. We wish to estimate the regression function m (x) = E (Y X = x) by mₙ (x), a function of x and (X₁, Y₁), , (Xₙ, Yₙ). For large classes of kernel estimates and nearest neighbor estimates, sufficient conditions are given for E\|mₙ (x) - m (x) |ᵖ\ 0 as n, almost all x. No additional conditions are imposed on the distribution of (X, Y). As a by-product, just assuming the boundedness of Y, the almost sure convergence to 0 of E\|mₙ (X) - m (X) \| X₁, Y₁, , Xₙ, Yₙ\ is established for the same estimates. Finally, the weak and strong Bayes risk consistency of the corresponding nonparametric discrimination rules is proved for all possible distributions of the data.