Asymptotic Behavior of M-Estimators of p Regression Parameters when p²/n is Large. I. Consistency

Consider the general linear model Y = xβ + R with Y and $R n$-dimensional, β p-dimensional, and X an n × p matrix with rows x'ᵢ. Let ψ be given and let β be an M-estimator of β satisfying 0 = ∑ xᵢψ(Yᵢ - x'ᵢβ). Previous authors have considered consistency and asymptotic normality of β when p is permitted to grow, but they have required at least p²/n → 0. Here the following result is presented: in typical regression cases, under reasonable conditions if p(log p)/n → 0 then \|β̂ - β\|² = Oₚ(p/n). A subsequent paper will show that β̂ has a normal approximation in Rᵖ if (p log p)3/2/n → 0 and that maxᵢ|x'ᵢ(β̂ - β)| →ₚ 0 (which would not follow from norm consistency if p²/n → ∞). In ANOVA cases, β̂ is not norm consistent, but it is shown here that max|x'ᵢ(β̂ - β)| →ₚ 0 if p log p/n → 0. A normality result for arbitrary linear combinations a'(β̂ - β) is also presented in this case.