On solving constrained optimization problems with neural networks: a penalty method approach

Deals with the use of neural networks to solve linear and nonlinear programming problems. The dynamics of these networks are analyzed. In particular, the dynamics of the canonical nonlinear programming circuit are analyzed. The circuit is shown to be a gradient system that seeks to minimize an unconstrained energy function that can be viewed as a penalty method approximation of the original problem. Next, the implementations that correspond to the dynamical canonical nonlinear programming circuit are examined. It is shown that the energy function that the system seeks to minimize is different than that of the canonical circuit, due to the saturation limits of op-amps in the circuit. It is also noted that this difference can cause the circuit to converge to a different state than the dynamical canonical circuit. To remedy this problem, a new circuit implementation is proposed.