The Bagged Median and the Bragged Mean

Bagging (bootstrap aggregating) is a procedure that aims at reducing the prediction error of a classifier or the mean square error of an estimate by averaging its values across bootstrap samples. We illustrate some of the effects of bagging on point estimation using only averages and medians. Our examples show that when we compute the bagged version of a robust estimate, the size of the bootstrap samples can be viewed as a tuning constant that controls the trade-off between efficiency and robustness. To quantify the robustness properties of bagged estimates we introduce a new concept of breakdown point that is useful in situations when resampling is needed. Finally, a robust version of bagging applied to the average leads to generalizations of previous results about the Hodges–Lehmann estimate.