On weighted Hochberg procedures

We consider different ways of constructing weighted Hochberg-type step-up multiple test procedures including closed procedures based on weighted Simes tests and their conservative step-up short-cuts, and step-up counterparts of two weighted Holm procedures. It is shown that the step-up counterparts have some serious pitfalls such as lack of familywise error rate control and lack of monotonicity in rejection decisions in terms of p-values. Therefore an exact closed procedure appears to be the best alternative, its only drawback being lack of simple stepwise structure. A conservative step-up short-cut to the closed procedure may be used instead, but with accompanying loss of power. Simulations are used to study the familywise error rate and power properties of the competing procedures for independent and correlated p-values. Although many of the results of this paper are negative, they are useful in highlighting the need for caution when procedures with similar pitfalls may be used. Copyright 2008, Oxford University Press.