Parametric Robustness: Small Biases can be Worthwhile

We study estimation of the parameters of a Gaussian linear model M₀ when we entertain the possibility that M₀ is invalid and a larger model M₁ should be assumed. Estimates are robust if their maximum risk over M₁ is finite and the most robust estimate is the least squares estimate under M₁. We apply notions of Hodges and Lehmann (1952) and Efron and Morris (1971) to obtain (biased) estimates which do well under M₀ at a small price in robustness. Extensions to confidence intervals, simultaneous estimation of several parameters and large sample approximations applying to nested parametric models are also discussed.