On the Power of Statistical Tests in the American Educational Research Journal

It is almost universally accepted by educational researchers that the power (probability of rejecting Ho when Ho is false, that is, 1-0-) of a statistical test is important and should be substantial. What is not universally accepted or known is that the power can and should be calculated and reported for every standard statistical test. The power of statistical tests already conducted in educational research is equally unknown. It is the purpose of this paper to report the level of power for recent statistical tests reported in the AERJ and to propose alternative reporting schemes relative to hypothesis testing to include power and effect size as well as the traditional a. Cohen (1962, 1969), Tversky and Kahnman (1971), Overall (1969) and others argue quite strongly that explicit computation of power relative to reasonable hypotheses should be made before any study is completed and subsequently reported. Tversky and Kahnman (1971) suggest three reasons why this computation is important: (1) Such computations can lead the researcher to the conclusion that there is no point in running the study unless the sample size is materially increased; (2) The computation is essential to the interpretation of negative results, that is, failures to reject the null hypothesis; and (3) Computed power gives the researcher an indication of the level of the probability of a valid rejection of the null hypothesis.