Estimating ???Individual Differences??? In the Response to an Experimental Treatment

713 Traditional repeated-measures analysis of the effect of an experimental treatment produces an estimate of the mean size of the effect but ignores the important possibility of variation in the size of the effect between subjects(“individual differences” in the response to the treatment). Such variation can now be estimated as a standard deviation of the effect by including it in a “mixed” model of fixed and random effects in the analysis. Our simulations show that: (a) a non-zero standard deviation effectively reduces the reliability of the outcome measure, so a larger sample size is needed to derive an acceptable confidence interval for the mean effect; (b) an acceptable confidence interval for the standard deviation can be obtained with a realistic sample size only in experiments with at least two pre or two post measurements; (c) the sample size is typically twice that required to give an acceptable confidence interval for the mean effect; (d) when included in the mixed model, a covariate that accounts for the standard deviation restores reliability and thereby produces acceptable confidence intervals for the mean effect and itself with a normal sample size, even in experiments with single pre and post measurements. Results (a)-(c) make individual differences a problem for researchers, but (d) provides a solution: assaying subject characteristics that are predictors of individual differences and including them in the analysis as covariates will improve the estimation of the mean response to a treatment and will identify the magnitude and origin of individual differences in the response.