Statistical Significance Tests: Equivalence and Reverse Tests Should Reduce Misinterpretation

Because both biological phenomena and the measurements researchers make in studying them are often quite variable, methods of statistical analysis have much to offer science. Statistical analysis has many aspects, including describing distributions of data, exploring the fit of data to various models, estimating and determining confidence intervals for parameters such as means and variances, and helping to identify real effects in the face of random variation (separating signal from noise). Significance tests, a tool for the latter purpose, are the focus of this article. Sometimes it seems that many biologists view significance testing as the overriding purpose of statistics, although many statisticians (Chatfield 1985) lament the overemphasis on that aspect of their trade at the expense of other aspects. To view significance tests at their most useful, consider the following scenario: An inebriated guest at a party claims the ability to influence the outcome of coin flips. Upon your challenge for evidence, he suggests that you and he go to separate rooms, with each of you accompanied by a witness of your choosing. You are to wait 5 minutes while he writes down a sequence of 10 “H” and “T” letters and then flip a coin (supplied by you) 10 times and record the outcomes. During the wait, you decide you will analyze the results with a significance test, with the null hypothesis being that he is fundamentally no better than a 50:50 guesser, and the one-tailed alternative being that he really has at least some ability to match more than 50% of coin tosses in this way.