Answers are grounded in this paper and cite their sources.
Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisingly large errors indefinitely, and therefore ask: Is there any stepsize schedule for gradient descent that accelerates the classic O(1/T) convergence rate, at any stopping time T?
No takes yet. Share an insight, caveat, or question.
Kornowski et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: