Investigating Smooth Multiple Regression by the Method of Average Derivatives

Abstract Let (x 1, …, xk, y) be a random vector where y denotes a response on the vector x of predictor variables. In this article we propose a technique termed average derivative estimation (ADE) for studying the mean response m(x) = E(y | x) through the estimation of the k vector of average derivatives δ = E(m′). The ADE procedure involves two stages: first estimate δ using an estimator , and then approximate m(x) by ), where ĝ is an estimator of the univariate regression of y on . We argue that the ADE procedure exhibits several attractive characteristics: data summarization through interpretable coefficients, graphical depiction of the possible nonlinearity between y and , and theoretical properties consistent with dimension reduction. We motivate the ADE procedure using examples of models that take the form . In this framework, δ is shown to be proportional to β and mcirc(x) infers m(x) exactly. The focus of the procedure is on the estimator , which is based on a simple average of kernel smoothers and is shown to be a √N consistent and asymptotically normal estimator of δ. The estimator ĝ(·) is a standard kernel regression estimator and is shown to have the same properties as the kernel regression of y on x Tδ. In sum, the estimator δ converges to δ at the rate typically available in parametric estimation problems, and mcirc(x) converges to E(y | xTδ) at the optimal one-dimensional nonparametric rate. We also give a consistent estimator of the asymptotic covariance matrix of δ, to facilitate inference. We discuss the conditions underlying these results, including how √N consistent estimation of δ requires undersmoothing relative to pointwise multivariate estimation. We also indicate the relationship between the ADE method and projection pursuit regression. For illustration, we apply the ADE method to data on automobile collisions.