Inference from Empirical Research.

ABSTRACT: Many researchers describe themselves as Bayesians in that they revise their prior beliefs based on observed empirical evidence. However, most studies are designed and reported as classical hypothesis tests, and research design issues are typically considered as determinants of abstract properties of statistical tests. Thus, although the primary function of empirical research is to influence beliefs, research design Issues are seldom considered in their fundamental role as determinants of beliefs. In this paper, a Bayesian perspective is used to analyze the role of basic properties of hypothesis tests in the revision of beliefs. Two main points are emphasized. First, hypothesis tests with low power are not only undesirable ex ante (because of the low probability of observing significant results) but also ex post (because little probability revision should be induced even when significant results am observed). Second, irrespective of the usual issues of statistical and methodological validity, the effective level of tests in published research is likely to exceed the stated level, thus reducing the amount of probability revision justified by reported results. In combination, these conclusions are especially troublesome. If tests reported in the accounting literature are characterized by both low power and high effective levels, the results of published tests properly have little or no impact on the beliefs of a Bayesian. The Bayesian framework is useful in understanding and analyzing the tradeoffs which are an inherent part of empirical research. The analysis here Identifies a Bayesian motivation for the common recommendations that researchers should attempt to maximize power in the design and execution of empirical tests and attempt to maintain the effective level of tests at their stated levels. Further, the analysis demonstrates the importance of explicit...