We consider the gradient method xₜ₊₁=xₜ+ₜ(sₜ+wₜ), where sₜ is a descent direction of a function f:→ and wₜ is a deterministic or stochastic error. We assume that f is Lipschitz continuous, that the stepsize ₜ diminishes to 0, and that sₜ and wₜ satisfy standard conditions. We show that either f(xₜ)→-∞ or f(xₜ) converges to a finite value and f(xₜ)→0 (with probability 1 in the stochastic case), and in doing so, we remove various boundedness conditions that are assumed in existing results, such as boundedness from below of f, boundedness of f(xₜ), or boundedness of xt.
No takes yet. Share an insight, caveat, or question.
Bertsekas et al. (2000) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: