Two Guidelines for Bootstrap Hypothesis Testing

Two guidelines for nonparametric bootstrap hypothesis testing are highlighted. The first recommends that resampling be done in a way that reflects the null hypothesis, even when the true hypothesis is distant from the null. The second guideline argues that bootstrap hypothesis tests should employ methods that are already recognized as having good features in the closely related problem of confidence interval construction. Violation of the first guideline can seriously reduce the power of a test. Sometimes this reduction is spectacular, since it is most serious when the null hypothesis is grossly in error. The second guideline is of some importance when the conclusion of a test is equivocal. It has no direct bearing on power, but improves the level accuracy of a test.