Lower Bounds for Contamination Bias: Globally Minimax Versus Locally Linear Estimation

We study how robust estimators can be in parametric families, obtaining a lower bound on the contamination bias of an estimator that holds for a wide class of parametric families. This lower bound includes as a special case the bound used to establish that the median is bias minimax among location equivariant estimators, and it is tight or nearly tight in a variety of other settings such as scale estimation, discrete exponential families and multiple linear regression. The minimum variation distance estimator has contamination bias within a dimension-free factor of this bound. A second lower bound applies to locally linear estimates and implies that such estimates cannot be bias minimax among all Fisher-consistent estimates in higher dimensions. In linear regression this class of estimates includes the familiar M-estimates, GM-estimates and S-estimates. In discrete exponential families, yet another lower bound implies that the "proportion of zeros" estimates has minimax bias if the median of the distribution is zero, a common situation in some fields. This bound also implies that the information-standardized sensitivity of every Fisher consistent estimate of the Poisson mean and of the Binomial proportion is unbounded.