Diagnostics for assumptions in moderate to large simple clinical trials: do they really help?

In this article, primarily we look at a case study, where prior to conducting the major efficacy analysis, one performs a diagnostic test for assumptions, and acts upon the result if the diagnostic test rejects the assumptions. Specifically, we show by an example that a hybrid approach of using a diagnostic test for equality of variance in a two-sample t-test situation can adversely affect, rather than protect, the operating characteristics of the study. If this kind of hybrid approach fails in such a simple setting, analysts should be very cautious about using hybrid approaches in more complex analyses of efficacy. Secondarily, we present rationale as to why the classical tests (or slightly modified versions) can be viewed as asymptotically non-parametric, and can actually be more robust against failure of assumptions than rank tests. Readers are cautioned that this illustration is limited to efficacy analysis, and is not meant as a criticism of other analyses, such as modelling or exploratory ones.