Significance tests have long been controversial in sociology and in the social sciences (e.g., Morrison and Henkel 1970). Despite critiques of them, they still are a near universal feature of quantitative analyses in sociology. Berk, Western, and Weiss (1995) (hereafter BWW) have reopened this issue in the context of what they call These are datasets that are nonreplicable such as data for all SMSAs, all states in the United States, or all nations in the developing world. Most of us who work with such samples have not adequately addressed the ambiguity in the meaning of significance tests in such contexts. BWW have done us a service by forcing us to rethink this issue. BWW have two major parts to their paper. One is a critique of the rationales and current practices of descriptive analysis and significance testing in apparent populations. The second part suggests that Bayesian analysis is a superior alternative. Most of my comments concentrate on the first part, though I do have some observations on their Bayesian proposal. My comments are directed at a number of claims in their paper: (1) there are fatal flaws in the justifications for significance testing in apparent populations, (2) the super population or randomization models are the only rationales for significance tests, (3) treating an apparent population as a true population implies a deterministic world, (4) apparent populations are not replicable, (5) the examples that they give can be characterized as apparent populations, and (6) a Bayesian approach has far fewer problems than the alternatives. The sections that follow address each of these.
No takes yet. Share an insight, caveat, or question.
Kenneth A. Bollen (1995) studied this question.