Variations on variable-metric methods. (With discussion)

In unconstrained minimization of a function f f , the method of Davidon-FletcherPowell (a "variable-metric" method) enables the inverse of the Hessian H H of f f to be approximated stepwise, using only values of the gradient of f f . It is shown here that, by solving a certain variational problem, formulas for the successive corrections to H H can be derived which closely resemble Davidon’s. A symmetric correction matrix is sought which minimizes a weighted Euclidean norm, and also satisfies the "DFP condition." Numerical tests are described, comparing the performance (on four "standard" test functions) of two variationally-derived formulas with Davidon’s. A proof by Y. Bard, modelled on Fletcher and Powell’s, showing that the new formulas give the exact H H after N N steps, is included in an appendix.