Forgetting-factor regrets for distributed second-order online optimisation

Most existing distributed algorithms use first-order gradient information, which is simple but converges slowly. This paper proposes two novel distributed Newton-step algorithms that leverage second-order curvature information to significantly improve convergence speed for dynamic optimisation problems. Over an undirected connected network, one algorithm uses the Hessian matrix, while the other exploits first-order information with an exponential decay to build a positive definite matrix. By adopting the above design, the second method not only circumvents the computational overhead of the Hessian matrix of the loss function but also enhances responsiveness to the most up-to-date information. Then, the Distributed Forgetting-Factor Regret (DFFR) of two algorithms are analysed. Moreover, sufficient conditions under which the algorithms can track the dynamic optimal decisions are rigorously derived. Numerical simulations show the superior performance of the proposed methods, especially over representative first-order algorithms, highlighting the advantages of second-order approaches.