Ridging out many covariates

The article considers a conditionally heteroskedastic linear regression setup with few regressors of interest and many nuisance covariates. We propose to subject the parameters corresponding to those nuisance covariates to a generalized ridge shrinkage. We show that under the assumption of dense random effects from the nuisance covariates, the ridge-out estimator of the parameters of interest is conditionally unbiased, and we derive the optimal ridge intensity that delivers conditional efficiency. When tight structures on the variance of random effects are imposed, the asymptotic variance of the ridge-out estimator, under the dimension asymptotics, may be arbitrarily smaller than that of the least squares estimator. We also demonstrate how the optimal ridge-out estimator can be implemented under tight structures on the variance of random effects and run simulation experiments where significant efficiency gains are possible to reach.