Analysis of monotone gradient methods

The gradient method is one simple method in nonlinear optimization.In this paper, we give a brief review on monotone gradient methods andstudy their numerical properties by introducing a new technique oflong-term observation. We find that, one monotone gradient algorithmwhich is proposed by Yuan recently shares with the Barzilai-Borwein (BB)method the property that the gradientcomponents with respect to the eigenvectors of the function Hessian aredecreasing together. This might partly explain why this algorithm by Yuanis comparable to the BB method in practice. Some examples are alsoprovided showing that the alternate minimization algorithm and theother algorithm by Yuan may fall into cycles. Some more efficientgradient algorithms are provided. Particularly, one of them is monotoneand performs better than the BB method in the quadratic case.