Learning in Neural Networks: Solvable Dynamics

The performance of an iterative learning rule for neural networks of spin-glass-type is studied. The algorithm minimizes a cost function quadratic in the synaptic couplings. An exact expression for the time development of the cost function is derived for the case of extensively many random patterns in large networks. A learning time as a function of the storage ratio α (number of patterns/number of spins) is calculated. It diverges as (1 − α)−2 as the storage ratio approaches 1.