This paper proves that stochastic gradient descent achieves tight convergence bounds in stochastic optimization for strongly convex objectives, indicating improved performance.
Stochastic optimization for strongly convex objectives is a fundamental problem in statistics and optimization. This paper revisits the standard Stochastic Gradient Descent (SGD) algorithm for strongly convex objectives and establishes tight uniform-in-time convergence bounds. We prove that with probability larger than $1 - β$, a log log k + log (1/β)/k convergence bound simultaneously holds for all k ∈ N₊, and show that this rate is tight up to constants. Our results also include an improved last-iterate convergence rate for SGD on strongly convex objectives.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: