Statistical Power Lost and Statistical Power Regained: The Bonferroni Procedure in Exploratory Research

The Bonferroni procedure controls the risk of rejecting one or more true null hypotheses no matter how many significance tests are performed, but permits the risk of failing to reject false null hypotheses to grow with the number of tests. The loss of statistical power associated with the use of this procedure is demonstrated, and two options for alleviating the problem are explored. Setting a less stringent significance level for the set of tests is shown to be less effective than increasing the sample size.