Stability, Monotonicity, and Necessity Analysis of Gradient Method with Penalty Terms for Dynamic Ridge Polynomial Neural Network

In recent years, high-order neural networks (HONNs) have gained widespread usage due to their excellent approximation ability, incorporating both ∑ neurons and ∏ neurons. The dynamic ridge polynomial neural network (DRPNN), as a type of HONN, has been highly valued. However, during the training process of the DRPNN, slow convergence occurs due to the presence of unknown factors. Given this, we propose a method to enhance the DRPNN convergence speed by incorporating a penalty term into the error function and training it using the gradient method. The proposed method significantly improves the convergence speed during network training. To explain the training characteristics of the proposed method, three theorems, namely stability, monotonicity, and necessity, are theoretically proven. More precisely, the whole network is stable over time based on the ultimate training results. The loss function monotonically decreases. Regarding the necessity, the ultimate training results must be an optimal solution. Furthermore, under the condition of the same iteration steps and error accuracy, experiments concerning function simulation and image simulation are advanced to verify the theoretical effectiveness of the proposed method, with the advanced analysis of the corresponding generalizability being discussed.