A Relationship between the BFGS and Conjugate Gradient Algorithms and Its Implications for New Algorithms

Based upon analysis and numerical experience, the BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm is currently considered to be one of the most effective algorithms for finding a minimum of an unconstrained function, f(x),x ∈ Rⁿ. However, when computer storage is at a premium, the usual alternative is to use a conjugate gradient (CG) method. In this paper we show that the two algorithms are related to one another in a particularly close way. Based upon these observations a new family of algorithms is proposed.