Comparing the Means of Two Independent Groups

Abstract Recently, CRESSIE and WHITFORD (1986) showed that Welch's test of H 0 :μ 1 = μ 2 can be biased, under nonnormality, where μ 1 and μ 2 are the means of two independent treatment groups. They suggested, therefore, that a two‐sample analog of Johnson's test be used instead. One goal in this paper is to examine the extent to which a two‐sample analog of Johnson's test improves upon Welch's technique in terms of Type I errors and power when sample sizes are small or moderately large. Several alternative procedures are also considered including an additional modification of Johnson's procedure, a procedure suggested by DUNNETT (1982) that uses Tiku's modified maximum likelihood estimate with 10% trimming, two versions of Efron's bootstrap, and a test recently proposed by WILCOX (1989). The paper also describes situations where Welch's procedure is not robust in terms of Type I errors. This is important because based on results currently available, Welch's procedure is thought to be nonrobust in terms of power, but robust in terms of Type I errors.