Statistical Inference When Classroom Quality is Measured With Error

Abstract A dramatic shift in research priorities has recently produced a large number of ambitious randomized trials in K-12 education. In most cases, the aim is to improve student academic learning by improving classroom instruction. Embedded in these studies are theories about how the quality of classroom must improve if these interventions are to succeed. The problem of measuring classroom quality then emerges as a major concern. This article first considers how errors of measurement reduce statistical power in studies of the impact of interventions classroom quality. We show how to use information about reliability to compute power and plan new research. At the same time, errors of measurement introduce bias into estimates of the association between classroom quality and student outcomes. We show how to use knowledge about the magnitude of measurement error to eliminate or reduce this bias. We also briefly review research on the design of studies of the reliability of classroom measures. Such studies are essential to evaluate promising new classroom interventions.