Simple Transformation Techniques for Improved Non‐parametric Regression

We propose and investigate two new methods for achieving less bias in non‐ parametric regression. We show that the new methods have bias of order h 4 , where h is a smoothing parameter, in contrast to the basic kernel estimator’s order h 2 . The methods are conceptually very simple. At the first stage, perform an ordinary non‐parametric regression on { x i , Y i } to obtain m^ ( x i ) (we use local linear fitting). In the first method, at the second stage, repeat the non‐parametric regression but on the transformed dataset { m^ ( x i , Y i )}, taking the estimator at x to be this second stage estimator at m^ ( x ). In the second, and more appealing, method, again perform non‐parametric regression on { m^ ( x i , Y i )}, but this time make the kernel weights depend on the original x scale rather than using the m^ ( x ) scale. We concentrate more of our effort in this paper on the latter because of its advantages over the former. Our emphasis is largely theoretical, but we also show that the latter method has practical potential through some simulated examples.