On the “Where” of Social Experiments: The Nature and Extent of the Generalizability Problem

Although randomized experiments are lauded for their high internal validity, they have been criticized for the limited external validity of their results. This chapter describes research strategies for investigating how much nonrepresentative site selection may limit external validity and bias impact findings. The magnitude of external validity bias is potentially much larger than what is thought of as an acceptable level of internal validity bias. The chapter argues that external validity bias should always be investigated by the best available means and addressed directly when presenting evaluation results. These observations flag the importance of making external validity a priority in evaluation planning.