Comparison of optimization algorithms

The measured convergence rates of several algorithms belonging to the class of 1 gradient methods in function space ’ are compared. The steepest descent, conjugate gradient and quasi-Newton algorithms, and the effect of different types of linear search routine and initial guess parameters on their convergence rates, are considered. It is concluded that conjugate gradient methods do offer significant advantages, even for non-linear system models and bounded controls.