A Resampling Procedure for Complex Survey Data

Abstract In complex survey data, often the sampling design induces a non-iid structure to the data (e.g., without replacement sampling, stratification, multistage, or unequal probability of selection). Though techniques for variance estimation and confidence intervals do exist, they often are cumbersome to implement or do not extend to complex designs. It would be desirable to have resampling methods that reuse the existing estimation system repeatedly, using computing power to avoid theoretical work, and that can be applied to such data. In recognition of this need, various resampling procedures for variance estimation and confidence intervals in sample survey data (where the sampling is without replacement) have been proposed in the literature. These include the jackknife, the with-replacement bootstrap (BWR), the without-replacement bootstrap (BWO), and the rescaling bootstrap. The BWR and BWO are applicable only to simple sampling designs. Others have shown the asymptotic consistency of jackknife variance estimates for nonlinear functions of means for multistage designs in which the primary sampling units are selected with replacement. However, when the primary sampling units are selected without replacement, the jackknife has been developed only for stratified sampling. The rescaling bootstrap extends to more complex sampling designs, but is applicable only to functions of means and can be computationally more intensive and difficult to use. The resampling method developed in this article retains the desirable properties of the BWR and BWO, but extends to more complex without-replacement sampling designs: (a) stratified random sampling, (b) two-stage cluster sampling, and (c) the Rao-Hartley-Cochran method of unequal probability sampling. The method consists of resampling without replacement from the data vector to mirror the original sampling design and then repeating this with replacement to match the usual variance estimates in the linear case. In the simple sampling designs to which the BWR is applicable, this resampling method contains the BWR as a special case and consequently suggests an extension of the BWR to two-stage cluster sampling. The properties of the variance estimators of and the bootstrap-t confidence intervals for θ are studied. The variance estimator is shown to be consistent when is a nonlinear function of means, in situation (a). The confidence intervals are shown to capture the second-order term of the Edgeworth expansion of the distribution of in situation (a) and thus account for skewness in the distribution. A simulation study for situation (a) suggests that the confidence intervals from the method track the nominal one-tailed error rates better than do the normal theory intervals, but the variance estimates are less stable. For the median, the method generally performs very well.