Key points are not available for this paper at this time.
In this evaluation we used Messick's construct validity as a conceptual framework for an empirical study assessing the validity of use, utility, and impact of office discipline referral (ODR) measures for data-based decision making about student behavior in schools. The Messick approach provided a rubric for testing the fit of our theory of use of ODR measures with empirical data on reported and actual use. It also facilitated our demonstration of Messick's principle that validation is both a developmental and an ongoing collaborative process among developers of educational and psychological measures, researchers interested in theories underlying such measures, and educators who use these measures in professional practice. We used a single-group, nonexperimental evaluation design to survey users of ODR measures from the standardized School Wide Information System in 22 elementary and 10 middle schools; respondents included school staff involved exclusively with data entry and staff actively involved in data-based decision making. Results were highly consistent across 2 independent data sources—electronic database records of actual access of summaries of ODR measures and self-report survey responses regarding frequencies and types of uses of ODR measures for decision making. Results indicated that ODR measures are regularly used for a variety of types of data-based decision making and are regarded as both efficient and effective for those purposes. We discuss implications of our SWIS ODR validity evaluation results within the context of the Messick framework.
Irvin et al. (Sun,) studied this question.