Comparison of approaches to variance estimation in non-probability samples

Non-probability surveys are becoming popular because they are easy to deploy and inexpensive. However, they frequently suffer from significant selection biases stemming from various underlying factors. These biases can result in unreliable estimates of the target population parameters. Conventional estimation techniques are inapplicable, as they require conditions that are not satisfied by non-probability survey data. Survey researchers have provided various methods to address selection bias. Among the most important methods are inverse probability weighting, mass imputation, and doubly robust methods. The estimators of the parameter of interest must be accompanied by their associated measures of accuracy. This implies the calculation of the variance of the estimator. We examine various factors that should inform the selection of an appropriate variance estimator when quantifying the precision of point estimates derived from key methods designed to address selection bias. Analytical variance expressions have been developed for the most widely used estimators; however, their precision depends on conditions that may not hold in real-world scenarios (such as transportability and ignorability) and on the use of linear models as predictive tools. Additionally, certain resampling-based methods have been developed for specific estimators, which also remain dependent on the aforementioned conditions. In this work, we present a comprehensive review of the methods developed to date for variance estimation of the most prominent estimators in non-probability sampling inference. Furthermore, we propose two new methodologies based on the bootstrap technique: the Bootstrap With Replacement (BWR) and the Bootstrap Without Replacement (BWOR). The first is a generalization of currently developed methods, which assume an independent and identically distributed bootstrap for both samples. The second is entirely novel, as it accounts for the sampling designs used to obtain both samples through a plug-in approach. An extensive simulation study, comprising both synthetic populations and real-world data setups, is conducted to evaluate their efficacy against analytical expressions. Our benchmark evaluations reveal that the proposed double bootstrap strategies provide a significant breakthrough over classical analytical expressions; in particular, the BWOR method delivers optimal performance, minimizing relative bias while maintaining excellent confidence interval coverage.