Answers are grounded in this paper and cite their sources.
This analysis uncovers optimal rates in stochastic optimization under Markovian noise, indicating improved methods for overparameterized models.
This paper investigates stochastic optimization problems under Markovian noise and the strong growth condition, motivated by overparameterized ML models. We present an improved analysis of the Accelerated Gradient Descent algorithm from [1] in the strongly convex case, showing that in low-noise regimes, the effect of Markovianity can be ignored. Furthermore, we derive the first lower bound that simultaneously depends on the Markov chain’s mixing time and the problem’s noise level, establishing the near-optimality of our results.
No takes yet. Share an insight, caveat, or question.
Chebykin et al. (2025) studied this question.