Methodological commentary: Effect of measurement error on tests of statistical significance

Using the domain-sampling model from classical test theory, the effects of measurement error on statistical tests for the difference between an obtained mean and a hypothesized mean, and the difference between two means, are demonstrated. The results indicate that lowering the reliability (i.e., increasing measurement error) of dependent variable data increases the chance of obtaining a nonsignificant result when a significant result is the correct outcome. Lowering the reliability also produces reduced estimates of strength of association.