Investigating Smooth Multiple Regression by the Method of Average Derivatives

Let (X '<i,V^t ie a random variable where y denotes a response on the 1 K vector X of predictor variables.In this paper we propose a technique (termed ADE) for studying the mean response m(x)=E(y|x) through the estimation of the k-vector of average derivatives 5=E(m').The ADE procedure involves two stages: first estimate 5 using an estimator 6. and then estimate m(x) as m(x)=g(x 8).where g is an estimator of the univariate regression of y on x 5 .We argue that the ADE procedure exhibits several attractive characteristics: data summarization through interpretable coefficients, graphical depiction of T' the possible nonlinearlty between y and x 5 , and theoretical properties consistent with dimension reduction.We motivate the ADE procedure using T examples of models that take the form m(x)=g(x $).In this framework, 5 is shown to be proportional to p, and m(x) infers m(x) exactly.The focus of the procedure is on the estimator 5, which Is based on a simple average of kernel smoothers, and is shown to be a ^consistent and asymptotically normal estimator of S. The estimator g( .) is a standard kernel regression estimator, and is shown to have the same properties as the kernel T regression of y on x 5 .In sum. the estimator S converges to .5 at the rate typically available in parametric estimation problems, and m(x) converges at the optimal one-dimensional nonparametric rate.We study the ADE estimators using Monte Carlo analysis, using sample designs with k=4 predictor variables.The ADE estimators perform well in samples of size N=50 generated from a linear models, and samples of size N=100 generated by a highly nonlinear model.For the latter samples, the ADE procedure is seen to have desirable goodness-of-f it and data summarization features relative to a multivariate regression smoother.