De-Biasing The Lasso With Degrees-of-Freedom Adjustment

This paper studies schemes to de-bias the Lasso in a linear model y=Xβ+ε where the goal is to construct confidence intervals for a₀Tβ in a direction a₀, where X has iid N (0, Σ) rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a full range of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by Lasso. Let s₀ be the true sparsity. If Σ is known and the ideal score vector proportional to XΣ^-1a₀ is used, the unadjusted de-biasing schemes proposed previously enjoy efficiency if s₀ n^2/3. However, if s₀ n^2/3, the unadjusted schemes cannot be efficient in certain a₀: then it is necessary to modify existing procedures by a degrees-of-freedom adjustment. This modification grants asymptotic efficiency for any a₀ when s₀/p 0 and s₀ (p/s₀) /n 0. If Σ is unknown, efficiency is granted for general a₀ when s₀ pn+\s_Ω p{n, \|Σ^-1a₀\|₁ p\|Σ^{-1/2a₀\|₂ n}\}+ (s_Ω, s₀) p n0 where s_Ω=\|Σ^-1a₀\|₀, provided that the de-biased estimate is modified with the degrees-of-freedom adjustment. The dependence in s₀, s_Ω and \|Σ^-1a₀\|₁ is optimal. Our estimated score vector provides a novel methodology to handle dense a₀. Our analysis shows that the degrees-of-freedom adjustment is not needed when the initial bias in direction a₀ is small, which is granted under stringent conditions on Σ^-1. The main proof argument is an interpolation path similar to that typically used to derive Slepian's lemma. It yields a new _ error bound for the Lasso which is of independent interest.