Heterogeneity of variance in experimental studies: A challenge to conventional interpretations.

The presence of heterogeneity of variance across groups indicates that the standard statistical model for treatment effects no longer applies. Specifically, the assumption that treatments add a constant to each subjects development fails. An alternative model is required to represent how treatment effects are distributed across individuals. We develop in this article a simple statistical model to demonstrate the link between heterogeneity of variance and random treatment effects. Next, we illustrate with results from two previously published studies how a failure to recognize the substan-tive importance of heterogeneity of variance obscured significant results present in these data. The article concludes with a review and synthesis of techniques for modeling variances. Although these methods have been well established in the statistical literature, they are not widely known by social and behavioral scientists. Psychological researchers have tended historically to view heterogeneity of variance as a methodological nuisance, an un-welcome obstacle in the pursuit of inferences about the effects of treatments on means. In their discussion of variance hetero-geneity, standard texts concentrate on identifying conditions under which such heterogeneity can safely be ignored so that standard analyses of means may proceed. It is usually argued that heterogeneity can be ignored when statistical tests for means are robust to violation of the homogeneity assumption (Glass Hopkins, 1984, pp. 238-240; Hays, 1981, p. 287; Winer, 1971, pp. 37-39). When such violations cannot be ig-nored, analysts tend to assume heterogeneity must be elimi-nated. The primary strategy for eliminating heterogeneity is to find a transformation of the dependent variable that stabilizes treatment group variances, enabling retention of the homoge-