Ibsen’s Baths: Reactivity and Insensitivity (A Misapplication of the Treatment-Control Design in a National Evaluation)

This discussion examines the insensitivity of the design and conduct of a national evaluation study. fundamental problem with the evaluation is that it typifies a misapplication of the treatmentcontrol or experimental design in educational research. First, the application of this design to a population of dropouts is unintentionally unethical, denying dropouts the opportunity to come back into the social system by entering an alternative school. Second, there are significant methodological flaws in the application of this design. control group in this study is actually a negative treatment group or a reactive control group (Cook & Campbell 1979) composed of individuals refused participation in the program. claim that withholding treatments from a portion of the group will provide more accurate estimates of the treatment effect (as argued by Tukey, 1978, and Gilbert, McPeek, & Mosteller, 1978) is spurious in this case. Controlled trials can actually ensure a more inaccurate estimate of treatment effects. This discussion demonstrates how a control group does not provide the no-cause baseline information expected of it. A discussion of this fundamental paradigm in educational evaluation parallels the controversy surrounding Ibsen's baths in his play An Enemy of the People (1958). Dr. Stockmann, the play's protagonist, was a medical official of the town's baths. After discovering contaminants in the baths, Stockmann attempted to publish his findings. He encountered significant resistance from the townspeople, who derived their income from tourists frequenting the baths. I have encountered similar resistance to the discussion of the misuse of a paradigm fundamental to educational research. Dr. Stockmann believed that The whole of [our] flourishing municipal life derive[d] its sustenance from a lie (p. 627). author contends that a parallel exists in an evaluation that relies on information drawn from a misapplication of the treatment-control design. Stockmann's community and the evaluation community are blinded by different agendas. difference between the two is that Stockmann's findings centered around personal and economic motivational factors, while the misapplication of the experimental design in this case was the result of real world constraints and insensitivity. results, however, are the same: both forms of blindness and resistance to substantive criticism function at the expense of others (in each case the individuals they seek to assist). A brief examination of the real world constraints and views of the federal bureaucracy, the research corporation, and the education research establishment I am indebted to Lee J. Cronbach and my colleagues at RMC Research Corporation and Stanford University for their insightful criticism and generous assistance in the preparation of this manuscript.