Pseudo-marginal Metropolis–Hastings sampling using averages of unbiased estimators

Summary We consider a pseudo-marginal Metropolis–Hastings kernel Pₘ that is constructed using an average of m exchangeable random variables, and an analogous kernel Pₛ that averages $s<m$ of these same random variables. Using an embedding technique to facilitate comparisons, we provide a lower bound for the asymptotic variance of any ergodic average associated with Pₘ in terms of the asymptotic variance of the corresponding ergodic average associated with Pₛ. We show that the bound is tight and disprove a conjecture that when the random variables to be averaged are independent, the asymptotic variance under Pₘ is never less than $s/m$ times the variance under Pₛ. The conjecture does, however, hold for continuous-time Markov chains. These results imply that if the computational cost of the algorithm is proportional to m, it is often better to set $m=1$. We provide intuition as to why these findings differ so markedly from recent results for pseudo-marginal kernels employing particle filter approximations. Our results are exemplified through two simulation studies; in the first the computational cost is effectively proportional to m and in the second there is a considerable start-up cost at each iteration.