Reliability and Statistical Power: How Measurement Fallibility Affects Power and Required Sample Sizes for Several Parametric and Nonparametric Statistics

The relationship between reliability and statistical power is considered, and tables that account for reduced reliability are presented. A series of Monte Carlo experiments were conducted to determine the effect of changes in reliability on parametric and nonparametric statistical methods, including the paired samples dependent t test, pooled-variance independent t test, one-way analysis of variance with three levels, Wilcoxon signed-rank test for paired samples, and Mann-Whitney-Wilcoxon test for independent groups. Power tables were created that illustrate the reduction in statistical power from decreased reliability for given sample sizes. Sample size tables were created to provide the approximate sample sizes required to achieve given levels of statistical power based for several levels of reliability.