Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors

Abstract As scientific problems grow in complexity, there is a pressing need for robust and scalable computational methods for fitting high-dimensional statistical models. Variational Bayes (VB) provides an approximate alternative to traditional sampling-based Bayesian inference, often reducing computation time from days to hours or minutes. VB typically minimizes the Kullback-Leibler divergence via coordinate ascent under a mean-field assumption. Its performance can be highly sensitive to prior specifications, particularly in sparse high-dimensional regression with spike-and-slab priors. A significant limitation of standard VB is its tendency to produce poorly calibrated predictions; that is, the predicted values often exhibit a systematic bias relative to the observed outcomes, failing to accurately reflect the true conditional expectation. Motivated by this, we apply parameter expansion to VB and propose a sparse parameter-expanded VB (spexvb) algorithm that improves robustness to prior settings and enhances predictive calibration. Compared to standard VB, spexvb demonstrates significantly enhanced robustness to prior specifications, yielding consistently lower predictive error, improved variable selection accuracy, and more stable and accurate posterior estimates, particularly in sparse and high-dimensional settings. We evaluate its performance through extensive simulations and a real-world application, demonstrating the practical advantages of parameter expansion in variational inference for high-dimensional regression.