Memory efficient BFGS neural-network learning algorithms using MLP-network: a survey

This paper surveys various implementation of a memory efficient second order (Broyden, Fletcher, Goldfard and Shanno) BFGS training algorithms which includes novel optimal memory (OM) BFGS neural network training algorithm, proposed by the present authors, which optimises performance in relation to available memory. Simulation results using a control benchmark problems show that OM BFGS, which is mathematically equivalent to full memory (FM) BFGS training when there are no constraints on memory, have performance gain compared to other memory efficient BFGS training algorithms.