The Evaluation of Broad-Aim Programs: Experimental Design, Its Difficulties, and an Alternative

There is an approach to the evaluation of social action programs which seems so sensible that it has been accepted without question. The underlying assumption is that action programs are designed to achieve specific ends and that their success can be established by demonstrating cause-effect relationships between the pro­ grams and their aims. In consequence, the preferred research design is an experi­ mental one in which aspects of the situation to be changed are measured before and after implementation of the action program. To support the argument that the program is responsible for the observed changes, the anticipated effects may be measured simultaneously in a control situation that does not receive the program (Campbell & Stanley, 1966). This plausible approach misleads when the action programs have broad aims and take unstandardized forms. The term broad-aim program is intended to describe programs that hope to achieve nonspecific forms of change-for-the-better and which also, because of their ambition and magnitude, involve unstandardized, large-scale interventions and are evaluated in only a few sites. These characteristics have been shared by a