A new spectral conjugate gradient method for unconstrained optimization and its application in neural networks

This work introduces a new variation of the Hestenes and Stiefel nonlinear conjugate gradient (HS) method by combining the advantages of the spectral conjugate gradient method and the conjugacy condition of the quasi-Newton method.The proposed method incorporates inexact line searches and categorizing it as a descent method.By employing line searches that satisfy the Wolfe conditions, we establish sufficient descent properties and global convergence condition, assuming that the appropriate conditions are met.Additionally, we perform numerical experiments utilizing benchmark functions frequently used in optimization assignments to evaluate the effectiveness of the proposed method.The results demonstrate that our method outperforms the traditional HS method.Furthermore, we successfully implement the newly developed technique to train neural networks (NNs), demonstrating its practicality for non-traditional optimization tasks.