Answers are grounded in this paper and cite their sources.
This work introduces new stepsize schedules for gradient descent, improving convergence rates for smooth convex objectives.
This work considers stepsize schedules for gradient descent on smooth convex objectives. We extend the existing literature and propose a unified technique for constructing stepsizes with analytic bounds for an arbitrary number of iterations. This technique constructs new stepsize schedules by concatenating two stepsize schedules with fewer steps. Using this approach, we introduce two new families of stepsize schedules, achieving a convergence rate of O(n-log₂(√ 2+1)) with state-of-the-art constants for the objective value and gradient norm of the last iterate, respectively. Furthermore, our analytically derived stepsize schedules either match or surpass the existing best numerically computed stepsize schedules.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: