How Good Is Good Enough? A Multidimensional, Best-Possible Standard for Research Design

Recent years have seen a shift in methodological emphasis from the observable properties of a sample to its unobservable properties, that is, judgments about the process by which the data were generated. Considerations of research design have moved front and center. This article attempts to bridge discussions of experimental and quasi-experimental data and of quantitative and qualitative approaches, so as to provide a unified framework for understanding research design in causal analysis. Specifically, the author argues that all research designs aim to satisfy certain fundamental criteria, applicable across methods and across fields. These criteria are understood as desirable, ceteris paribus, and as matters of degree. The implications of this framework for methodological standards in the social sciences are taken up in the final section of the article. There, the author argues for a best-possible standard of proof that judges overall methodological adequacy in light of other possible research designs that might be applied to a particular research question.