An Algorithm, Based on Singular Perturbation Theory, for Ill-Conditioned Minimization Problems

The problem of finding an unconstrained local minimum of a function f is considered. Ill-conditioned minimization problems are characterized by long, very narrow valleys which tend to trap many numerical procedures, i.e., methods applied to such problems tend to “jam”. To overcome this difficulty, the problem is reformulated as that of integrating the differential equation x' = - ∇ f(x). The ill-conditioning of the minimization problem carries over to the differential equation by making it stiff. Thus methods for integrating stiff systems of equations are investigated and adapted to this special problem. In particular, a recent method of Miranker based on singular perturbation theory is modified and shown to be applicable to this problem. Some numerical results are presented which indicate that the resulting algorithms are quite competitive.