Comparison of Four Procedures for Multiple Comparisons Among Means (Pairwise Contrasts) for Arbitrary Sample Sizes

For the situation in which the contrasts of interest are limited to the pairwise comparisons among the means of K samples of equal or unequal sizes, four normal univariate single-stage multiple comparison procedures are compared for significance levels not exceeding 0.05: Scheffé's S-method, Dunn's (1, 2] and šidák's [17] improved version of the Bonferroni method, Hochberg's GT2 procedure [8] utilizing the maximum modulus, and Spjøvoll and Stoline's T′-method [19]. Rules are given for determining if any method is uniformly preferable (best for all contrasts). Nonuniform preference rules are also proposed and applied to some examples. Auxiliary tables are provided for selecting a method for significance levels 0.01 and 0.05 for several values of v, the number of degrees of freedom of an independent variance estimate, and K. It is shown that the T′-method is uniformly preferable when the sample sizes are “nearly” equal, while one of the other methods will be uniformly preferable when all sample sizes are “sufficiently” different. The comparison is readily extended to the case in which only a subset of the pairwise contrasts is of interest.