Using Ranks to Perform Exact and Estimated Exact Tests in Designed Experiments

A procedure is studied that uses rank transformed data to perform exact and estimated exact tests which is an alternative to the commonly used F-ratio test procedure. First, a common parametric test statistic is computed using rank transformed data, where two methods of ranking - ranks taken of the original observations, and ranks taken after aligning the observations - are studied. Significance is then determined using either the exact permutation distribution of the statistic or an estimate of this distribution based on a random sample of all possible permutations. Simulation studies compare the performance of this method to both the normal theory parametric F-test and the traditional rank transform procedure. Power and nominal type-I error rates are compared under conditions when normal theory assumptions are satisfied as well as when these assumptions are violated. The method is studied for a two factor factorial arrangement of treatments in a completely randomized design and also for a split-unit experiment.