Answers are grounded in this paper and cite their sources.
In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the conjugate gradient method is developed. The memory is used to both detect the loss of orthogonality and to restore orthogonality. An implementation of the algorithm is presented based on the CG DESCENT conjugate gradient method. Limited memory CG DESCENT (L-CG DESCENT) possesses a global convergence property similar to that of the memoryless algorithm, but much better practical performance. Numerical comparisons to the limited memory BFGS method (L-BFGS) are given using the CUTEr test problems.
No takes yet. Share an insight, caveat, or question.
Hager et al. (2013) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: