A new alternative to the ANOVA F and new results on James's second‐order method

A new procedure is proposed for testing the hypothesis that the means of J independent normal distributions are equal. Extensive simulations indicate that the new procedure always gives the experimenter more control over the probability of a Type I error than does the Brown–Forsythe or Welch adjusted degree of freedom techniques. For equal and reasonably large sample sizes the Welch procedure might be preferred, however, because it can have substantially more power, and it provides adequate though liberal control over Type I errors. However, Welch's procedure cannot always be recommended for reasons described in the paper. For unequal sample sizes both the Welch and Brown–Forsythe procedures are known to be unsatisfactory in certain practical situations, while the new procedure has Type I error probabilities that are much closer to the nominal level. Moreover, the new procedure seems to give conservative results provided the sample sizes in each group are greater than or equal to 10. New results are also reported on James's second order method. Although computationally more tedious, James's procedure is recommended for general use.