A Bayesian Approach to Ranking and Selection of Related Means with Alternatives to Analysis-of-Variance Methodology

Abstract A set of unknown normal means (treatment effects, say) θ1, θ2, …, θk is to be investigated. Two common questions in analysis of variance and ranking and selection are as follows: (a) What is the strength of evidence against the hypothesis H 0 of equality of means? (b) If H 0 is false, which mean is the largest (or smallest)? A Bayesian approach to the problem is taken, leading to calculation of the posterior probability of H 0 and the posterior probabilities that each mean is the largest, conditional on H 0 being false. A variety of exchangeable, nonexchangeable, informative, and noninformative prior assumptions are considered. Calculations involve, at worst, only low-dimensional numerical integration, in spite of the fact that the dimension k can be arbitrarily large. As an example, Table 1 presents, for each baseball team in the National League in 1984, the highest batting average obtained by any player on the team with at least 150 at bats. The observed batting averages are treated as sample proportions from binomial distributions with parameters θi = true probability of getting a hit for the given player, and it is desired to select the best hitter from the group, namely the player with the largest θi. Calculated, using a Bayesian model of exchangeability for the θi, are quantities such as the posterior probabilities that each θi is the largest. Such posterior probabilities give very easy to understand and useful measures to assist in selection and ranking. Of substantial interest is that, in unbalanced examples such as the baseball example (the players all had different numbers of “at bats, ” and hence different variances), it need not be the case that the treatment with the largest sample mean is judged to have the largest true mean. Thus Player 1's observed batting average was higher than that of Player 2, but Player 2 had a substantially smaller variance and was determined (by the hierarchical Bayes method) to have a larger probability of being the best true hitter. An interesting sidelight to the development is the presentation of a closed-form solution for testing H 0: θ1 = θ2 versus H 1: θ1 θ 2, when the treatments are judged to be a priori exchangeable.