8: Implications of Educational Evaluation for Evaluation Policy

The aim of this paper is to review certain policy implications of evaluative research. Our special interest lies in evaluation policy, rather than in the broad category of educational policy, which may be educed from research results. Of the several reasons for the choice of topic, not the least is a simple, perhaps morbid curiosity about what can be said about evaluation policy, taking account of recent large-scale efforts to estimate the effects of educational pro-grams. A second reason is rather more pragmatic. Policy does affect the quality of evaluative research (at least at times), and the United States lacks a coherent evaluation policy in education as it lacks such policy in health services and other areas. We do not pretend that this paper provides coherent policy. How-ever, we do believe the matters covered here are a legitimate part of any dura-ble evaluation policy. Part 1 of the paper deals with the recent history of evaluation including the instability of national interest in evidence, unfounded discouragement, and catholicity. Part 2 focuses on design of evaluations, especially its influence on the quality of information that is generated about an educational program. The implementation of new pilot programs and the independent execution of evalua-tion design are discussed in Part 3; policy implications of collective experience in analyzing results are discussed in Part 4. Part 5 deals with the availability of Support for research on which this paper has been based was provided by the National Institute of Education and the National Science Foundation. The opinions expressed here do not necessarily reflect the view of the sponsoring agencies. Robert Linn was kind enough to provide his reactions to an earlier draft. The paper is based partly on ideas