8: Analytic Methods in Studies of Educational Effects

The publication of Equality of Educational Opportunity (Coleman, Campbell, Hobson, McPartland, Mood, Weinfeld, Cain Coleman, 1968, 1970; Mosteller Smith, 1968) have underscored the lack of agreement among social scientists about analytic methods appropriate for the study of educational effects. Applying different analytic methods to the same data, researchers have often reached different, and occasionally contradictory, conclusions. It is not surprising that the general public and policymakers have not had confidence in the research findings of educational researchers. Moreover, the conflicting conclusions have enabled some policymakers to select those findings that are in accord with their own preconceptions and biases. While the widespread availability of computer facilities and package programs has liberated researchers from the drudgery of computations required by complex analytic methods, the relative ease with which they can now be executed has increased the danger of applying them blindly. Moreover, it seems, some researchers believe that the absence of theoretical formulations can be masked, or compensated for, by the use of complex and sophisticated analytic techniques. It is a truism that methods per se mean little unless they are integrated within a theoretical context and are used to test hypotheses derived from theory.