Evidence and Inference in the Comparative Case Study

Comparative case studies often rely on a practice known as selecting on the dependent variable. This technique involves choosing some phenomenon of political interest, gathering data on occurrences of the phenomenon, then determining what characteristics the occurrences have in common. A methodological strategy with a long history, selecting on the dependent variable has found numerous applications in political science, including the study of economic growth, social revolutions, and international conflict.' Selecting on the dependent variable has provided an important method for gathering information when there is relatively little data, though it is not limited to such situations. Its use has recently come under rather sharp attack. If observations are selected based on the value of the dependent variable, then estimates obtained by ordinary least squares will be biased. This bias can not be corrected by the introduction of control variables. Gathering more data will not solve the problem; the bias remains even if the number of observations goes to infinity. In short, inferences drawn from cases selected on the dependent variable are highly suspect.2 Impressive treatments of the problem from the position of comparative case study work have been presented by Barbara Geddes and by Gary King, Robert Keohane, and Sidney Verba.3 They have not only alerted the discipline to an important problem but have suggested concrete ways to deal with problems of selection bias.4 I would like to offer a qualified defense of analyses of a small number of cases selected on the dependent variable. As is well-known, selecting on the dependent variable is perfectly admissible if one is evaluating necessary (as opposed to sufficient) conditions.5 However, with a few exceptions, this argument has gone unnoticed in the debate on case selection.6 By concentrating on the logic of testing necessary conditions, this paper attempts to show that the methodological implications of selecting on the dependent variable are much less threatening to the validity of small-n comparative case study work than has often been claimed.