The Small Sample Behavior of Some Statistics Which Test the Equality of Several Means

Four statistics which may be used to test the equality of population means are com-pared with respect to their robustness under heteroscedasticity, their power, and the overlap of their critical regions. The four are: the ANOVA F-statistic; a modified F which has the same numerator as the ANOVA but an altered denominator; and two similar statistics proposed by Welch and James which differ primarily in their approximations for their critical values. The critical values proposed by Welch are a better approximation for small sample sizes than that proposed by James. Both Welch's statistic and the modified F are robust under the inequality of variances. The choice between them depends upon the magnitude of the means and their standard errors. When the population variances are equal, the critical region of the modified F more closely approximates that of the ANOVA than does Welch's.