Answers are grounded in this paper and cite their sources.
Theoretical analysis demonstrates geometric ergodicity in nonconvex stochastic gradient Langevin dynamics, highlighting convergence guarantees for sampling algorithms with constant step sizes.
We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact sets. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of [Formula: see text] distance. The generalization to non-gradient drifts is also included.
No takes yet. Share an insight, caveat, or question.
Li et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: