Rules of Evidence: A Larger Context for the Statistical Debate

Statistical tests define rules of evidence in psychological science Agreed-upon rules of evidence are needed to discuss what is true Statistics are probability statements about the reliability of associations In public and journalistic debates about psychological topics, the need for statistical rules is evident Anecdotes, intuitions, and sensational and unlikely events often dominate popular discussions Psychological science demands that the probability of events and associations be evaluated by statistical means Understanding of statistical tests can be improved by abandoning the term significance, which means reliability to psychologists, but in everyday English, means importance The importance of a result depends on magnitude of association, not on whether the event has a nonzero chance of recurring Given large samples and multiple measures, statistical tests are used too often to draw misleading conclusions about the importance of research results Better uses of statistics would focus on the magnitude of effects and error estimates