Comparing Weighting Strategies for Multiple Non-Probability Samples with an Application to American Indian Epidemiology

Non-probability samples are frequently used when a sampling frame is unavailable, or probability sampling is prohibitively costly, but naive estimates may be biased by selective participation. Most weighting research considers one non-probability sample and one probability reference sample. When several non-probability sources are available, analysts can either pool the sources and estimate one set of weights (Method 1) or estimate source-specific weights and then combine the weighted sources (Method 2). We formalize and compare these two implementation strategies for calibration and adjusted logistic propensity (ALP) weighting; the contribution is the comparison of integration order rather than a new base weighting estimator. Identification requires conditional ignorability of participation given shared covariates, positivity/common support, a valid probability reference sample, and adequate specification of the relevant calibration or participation model. In 2000 Monte Carlo replications with finite-population size N = 100,000, probability-sample size nB = 5000, expected source size approximately 500, K = 2, 5, or 10, simple-random-sampling or probability-proportional-to-size reference designs, and linear or nonlinear outcome structures, calibration had lower relative standard error (RSE) in all 24 matched comparisons and lower relative root mean squared error (RRMSE) in 23 of 24. Method 1 had lower RRMSE in 21 of 24 comparisons and in every nonlinear scenario, although it was not uniformly superior in the linear calibration settings. An application combining two American Indian/Alaska Native COVID-19 studies with the 2022 American Community Survey shows that integration order can materially affect prevalence estimates. Method 1 with calibration provided the most stable performance in the settings studied, but this recommendation is conditional on overlap and model adequacy.