In this paper, we propose a simple, fast and easy to implement algorithm lossgrad (locally optimal step-size in gradient descent), which automatically modies the step-size in gradient descent during neural networks training. Given a function f , a point x, and the gradient xf of f , we aim to nd the step-size h which is (locally) optimal, i.e. satises:
No takes yet. Share an insight, caveat, or question.
Wójcik et al. (2018) studied this question.
Synapse has enriched 5 related papers. Consider them for comparative context: