Variance heterogeneity, transformations, and models of species abundance: a cautionary tale

Ecological systems have intrinsic heterogeneity. Counts of abundances of species often show heterogeneity of variances among observational groups or populations. This is most often dealt with by using a transformation of the data followed by a traditional statistical analysis that requires homogeneity. Such an approach is extremely useful when the mean–variance relationship is consistent across the data set. In some situations, however, the mean–variance relationship does not stay constant, e.g., the degree of spatial aggregation of organisms can change in space and time. In these cases, transforming the data to "fix" the problem of heterogeneity can result in apparently grossly inflated type I error. The use of a transformation alters the model under test and also has an important effect on the spatial scale of the hypothesis. The use of nonparametric alternatives, such as permutation or bootstrap tests, does not solve this problem. Explicit models of these kinds of distributional changes, where they occur, are necessary.