This is the authors' abstract. We don't add key points for this paper.
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model). A key feature of GT is a tracking mechanism that allows us to overcome data heterogeneity between nodes. We develop a novel decentralized tracking mechanism, K-GT, which enables communication-efficient local updates in GT while inheriting the data-independence property of GT. We prove a convergence rate for K-GT on smooth non-convex functions and prove that it reduces the communication overhead asymptotically by a linear factor K, where K denotes the number of local steps. We illustrate the robustness and effectiveness of this heterogeneity correction on convex and non-convex benchmark problems and a non-convex neural network training task with the MNIST dataset.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Liu et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: