Sizing and Least-Change Secant Methods

Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845–862]. In this paper, the function \[ω (A) = ( {{{{{{trace}(A)} / n}}}{{{det}(A)^{{1 / n}} }}} )\] is shown to be a measure of change with a direct connection to the Oren–Luenberger strategy. This measure is interesting because it is related to the ₂ condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why the optimally conditioned updates are not significantly better than the BFGS update. The theory results in several new interesting updates including a self-scaling, hereditarily positive definite, update in the Broyden class which is not necessarily in the convex class. This update, in conjunction with the Oren–Luenberger scaling strategy at the first iteration only, was consistently the best in numerical tests.