nlTGCR: A Class of Nonlinear Acceleration Procedures Based on Conjugate Residuals

.This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods—depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.Keywordsnonlinear accelerationgeneralized conjugate residualtruncated GCRAnderson accelerationNewton's methoddeep learningMSC codes65F1068W2565F0890C53