Moving From the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength

Abstract As an example of how the effectiveness of well-designed laboratory interventions is often diffused in the field, we examined the effects of a motivation intervention on students' perceptions and learning. The intervention proved to be more effective in the laboratory (g = 0.45) than the field (g = 0.05) in enhancing subsequent motivation. We explored this reduction in treatment effectiveness through a fidelity analysis that examined the extent to which participants responded to the treatment. We calculated fidelity as three indices of achieved relative treatment strength (Cordray & Pion, 2006 Cordray, D. S. . The cause … or the ‘what’ of what works?. Paper presented at the Institute for Education Sciences 2006 Research Conference. Washington, DC. June, Google Scholar), and found that, regardless of how fidelity was calculated, achieved relative strength was about 1 standard deviation less in the classroom than the laboratory. In addition, greater levels of achieved relative strength were associated with greater differences in the motivational outcome—indicating that intervention was more effective for participants who actually received the treatment than those who did not. Multilevel analyses indicated that the drop in classroom treatment fidelity was partially because of teacher, rather than student, factors. Implications for theoretical models of change and research are discussed.