Link Between and Comparison and Combination of Zhang Neural Network and Quasi-Newton BFGS Method for Time-Varying Quadratic Minimization

Since 2001, a novel type of recurrent neural network called Zhang neural network (ZNN) has been proposed, investigated, and exploited for solving online time-varying problems in a variety of scientific and engineering fields. In this paper, three discrete-time ZNN models are first proposed to solve the problem of time-varying quadratic minimization (TVQM). Such discrete-time ZNN models exploit methodologically the time derivatives of time-varying coefficients and the inverse of the time-varying coefficient matrix. To eliminate explicit matrix-inversion operation, the quasi-Newton BFGS method is introduced, which approximates effectively the inverse of the Hessian matrix; thus, three discrete-time ZNN models combined with the quasi-Newton BFGS method (named ZNN-BFGS) are proposed and investigated for TVQM. In addition, according to the criterion of whether the time-derivative information of time-varying coefficients is explicitly known/used or not, these proposed discrete-time models are classified into three categories: 1) models with time-derivative information known (i.e., ZNN-K and ZNN-BFGS-K models), 2) models with time-derivative information unknown (i.e., ZNN-U and ZNN-BFGS-U models), and 3) simplified models without using time-derivative information (i.e., ZNN-S and ZNN-BFGS-S models). The well-known gradient-based neural network is also developed to handle TVQM for comparison with the proposed ZNN and ZNN-BFGS models. Illustrative examples are provided and analyzed to substantiate the efficacy of these proposed models for TVQM.