“Equivalent Sample Size” and “Equivalent Degrees of Freedom” Refinements for Inference Using Survey Weights under Superpopulation Models

Abstract A number of procedures have been proposed to attack different inference problems for data drawn from a survey with a complex sample design (i.e., a design that entails unequal weighting). Most procedures either are based on finite-population assumptions or require the specification of an explicit model using a superpopulation rationale. Herein we propose some relatively simple approximate procedures that are based on a superpopulation model. They provide valid variance estimators, test statistics, and confidence intervals that allow for sample design effects as expressed by design weights and other weights. The procedures do not rely on conditioning on model elements such as covariates to adjust for design effects. Instead, we obtain estimators by rescaling sample weights to sum to the equivalent sample size (equal to sample size divided by design effect). Using weighted estimators for superpopulation models, we obtain approximations to confidence bounds on the mean for simple sampling situations as well as for cluster sampling, post-stratification, and stratified sampling. We also obtain approximate tests of hypotheses for one-way analysis of variance and k × 2 homogeneity testing. For all of these, further refinements based on the concept of equivalent degrees of freedom are provided. Additionally, a general method for determining and using poststratification weights is described and illustrated. The procedures in this article are better justified than the common expedient of making proportional adjustments so that the weights add to the sample size. Key Words: Cluster samplingDesign effectHomogeneity testingOne-way variance analysisPoststratificationStratified sampling