Answers are grounded in this paper and cite their sources.
Nesterov's momentum trick is famously known for accelerating gradient descent, and has been proven useful in building fast iterative algorithms. However, in the stochastic setting, counterexamples exist and prevent Nesterov's momentum from providing similar acceleration, even if the underlying problem is convex.
No takes yet. Share an insight, caveat, or question.
Zeyuan Allen-Zhu (2017) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: