A Bayesian Significance Test for Multinomial Distributions

Summary A Bayesian significance test for multinomial distributions is discussed, together with the results of 18 numerical experiments in which the test is compared with non-Bayesian methods. Several different non-Bayesian criteria are considered because the circumstances under which their tail-area probabilities can be conveniently approximated differ from one to the other. A provisional empirical formula, connecting the Bayes factors with the tail-area probabilities, is found to be correct within a factor of 6 in the 18 experiments. As a by-product, a new non-Bayesian statistic is suggested by the theory, and its asymptotic distribution obtained. It seems to be useful sometimes when chi-squared is not, although chi-squared also has a Bayesian justification for large samples. The work originated in, and is relevant to, the problem of the estimation of multinomial probabilities, but significance tests are a better proving ground for assumptions concerning the initial (prior) distribution of the physical probabilities.