Sequential Design and Estimation in Heteroscedastic Nonparametric Regression

The paper considers, for the first time in the literature, sharp minimax design of predictors and sharp minimax sequential estimation of regression functions in a classical heteroscedastic nonparametric regression. The suggested methodology of a sharp minimax design of predictors in controlled regression experiments with fixed-size samples is based on minimization of the coefficient of difficulty of an underlying regression model, which is defined as a factor in changing the sample size that makes the estimation problem comparable with estimation in a homoscedastic regression with unit-variance errors and uniform design. It is established that an optimal design density is proportional to an underlying scale function. This makes a sequential design of predictors, based on a sequential choice of design densities, a particularly attractive statistical strategy that allows the statistician to minimize the coefficient of difficulty during a controlled experiment. It is shown that a developed sequential design of predictors asymptotically matches the mean integrated squared error (MISE) of an oracle that knows the optimal design (the scale function). Another considered problem is a sequential estimation of an underlying regression function with a preassigned MISE and sharp minimax mean stopping time in a heteroscedastic regression setting where predictors are generated according to some (in general unknown) underlying design density. For this setting the theory of a sharp minimax sequential estimation of a regression function is developed and a sharp minimax sequential estimator is suggested. Finally, a problem of sequential estimation of a regression function based on a sequential design is considered. A procedure of sequential analysis of heteroscedastic regression is suggested that matches the performance of an oracle that knows the optimal design density and optimal stopping time, which implies a preassigned MISE of estimation of an underlying regression function. A discussion of possible extensions and further developments is also presented.