Pooling Disparate Observations

Theory: Classical statistical inference takes as given the population governed by a posited statistical model and associated set of parameters. But social theories seldom include clear specifications of the populations to which they are supposed to be applicable, so data analysts frequently face difficult choices about which observations to include in their analyses. Hypotheses: Conventional approaches to selecting relevant observations are likely either to underexploit the available data (by discarding problematic observations that could provide some information about the parameters of interest) or to overexploit the available data (by estimating alternative models and interpreting the best results as though they were produced in accordance with the standard assumptions of classical statistical inference). Methods: I propose a technique, dubbed pooling, which provides a simple and coherent way either to incorporate prior beliefs about the theoretical relevance of disparate observations or to explore the implications of prior uncertainty about their relevance. The technique is easy to implement and has a plausible rationale in Bayesian statistical theory. Results: I illustrate the potential utility of fractional pooling by applying the technique to political data originally analyzed by Ashenfelter (1994), Powell (1982), and Alesina, Londregan, and Rosenthal (1993). These examples demonstrate that conventional approaches to analyzing disparate observations can sometimes be seriously misleading, and that the approach proposed here can enrich our understanding of the inferential implications of unavoidably subjective judgments about the theoretical relevance of available data.