Accelerating Gaussian Diffusions

Let π(x) be a given probability density proportional to exp(-U(x)) in a high-dimensional Euclidean space Rᵐ. The diffusion dX(t) = -∇ U(X(t))dt + √ 2 dW(t) is often used to sample from π. Instead of -∇ U(x), we consider diffusions with smooth drift $b(x)$ and having equilibrium π(x). First we study some general properties and then concentrate on the Gaussian case, namely, -∇ U(x) = Dx with a strictly negative-definite real matrix D and $b(x) = Bx$ with a stability matrix B; that is, the real parts of the eigenvalues of B are strictly negative. Using the rate of convergence of the covariance of $X(t)$ [or together with $EX(t)$] as the criterion, we prove that, among all such $b(x)$, the drift $Dx$ is the worst choice and that improvement can be made if and only if the eigenvalues of D are not identical. In fact, the convergence rate of the covariance is exp(2λM(B)t), where λM(B) is the maximum of the real parts of the eigenvalues of B and the infimum of λM(B) over all such B is 1/m tr D. If, for example, a "circulant" drift $(∂ U/∂ x_m - ∂ U/∂ x_2,∂ U/∂ x_1 - ∂ U/∂ x_3, ⋯, {∂ U}{∂ xₘ₋₁} - ∂ U/∂ x_1)$ is added to $Dx$, then for essentially all $D$, the diffusion with this modified drift has a better convergence rate.