Summary Likelihood-based Variational Inference (VI) methods have recently gained traction in various fields of geophysics as they can dramatically reduce the computational cost of estimating Bayesian posterior probability distributions of target parameter values compared with traditional approaches such as Markov chain Monte Carlo (McMC) methods. In many geophysical applications of these methods, the forward problem depends also on nuisance parameters – variables that affect the forward problem, but which are not of direct interest. However, the impact on posterior estimates of marginalising over nuisance parameters in likelihood-based VI approaches remains poorly understood in geophysics. Through analytical derivation and a low-dimensional numerical example, we demonstrate that standard implementations of likelihood-based VI methods systematically underestimate posterior uncertainty when nuisance parameters are either marginalised via integration or are fixed at their maximum likelihood estimates. This underestimation occurs because Jensen’s inequality is invoked to simplify the approximation of the marginal likelihood integral, which can bias interpretations of solutions. We further show that likelihood-free VI, in particular simulation-based inference (SBI), recovers the correct posterior distribution in our experiments while remaining more computationally efficient than McMC in low dimensional problems. Our work thus clarifies conceptual differences between likelihood-based and likelihood-free Variational Inference, and in particular the impact of the approximation invoked by Jensen’s inequality.
Dromigny et al. (Thu,) studied this question.