A learning algorithm for analog, fully recurrent neural networks

A learning algorithm for recurrent neural networks is derived. This algorithm allows a network to learn specified trajectories in state space in response to various input sequences. The network dynamics are described by a system of coupled differential equations that specify the continuous change of the unit activities and weights over time. The algorithm is nonlocal, in that a change in the connection weight between two units may depend on the values for some of the weights between different units. However, the operation of a learned network (fixed weights) is local. If the network units are specified to behave like electronic amplifiers, then an analog implementation of the learned network is straightforward. An example demonstrates the use of the algorithm in a completely connected network of four units. The network creates a limit cycle attractor in order to perform the specified task.>