Learning algorithm that gives the Bayes generalization limit for perceptrons

A variational approach to the study of learning a linearly separable rule by a single-layer perceptron leads to a gradient descent learning algorithm with exactly the same generalization ability as the Bayes limit calculated by Opper and Haussler [Phys. Rev. Lett. 66, 2677 (1991)]. This is done by finding, through the Gardner-Derrida replica method, the student-teacher overlap R as a functional of the algorithm cost function and maximizing this functional. The resulting cost function is closely related to the optimal cost function derived for on-line learning.