Valid and efficient inference for nonparametric variable importance in two-phase studies

We consider a common nonparametric regression setting, where the data consist of a response variable Y, some easily obtainable covariates X, and a set of costly covariates Z. Before establishing predictive models for Y, a natural question arises: Is it worthwhile to include Z as predictors, given the additional cost of collecting data on Z for both training the models and predicting Y for future individuals? Therefore, we aim to conduct preliminary investigations to infer importance of Z in predicting Y in the presence of X. To achieve this goal, we propose a nonparametric variable importance measure for Z. It is defined as a parameter that aggregates maximum potential contributions of Z in single or multiple predictive models, with contributions quantified by general loss functions. Considering two-phase data that provide a large number of observations for (Y,X) with the expensive Z measured only in a small subsample, we develop a novel approach to infer the proposed importance measure, accommodating missingness of Z in the sample by substituting functions of (Y,X) for each individual’s contribution to the predictive loss of models involving Z. Our approach attains unified and efficient inference regardless of whether Z makes zero or positive contribution to predicting Y, a desirable yet surprising property owing to data incompleteness. As intermediate steps of our theoretical development, we establish novel results in two relevant research areas, semi-supervised inference and two-phase nonparametric estimation. Numerical results from both simulated and real data demonstrate superior performance of our approach.