Answers are grounded in this paper and cite their sources.
In distributed optimization, a large number of machines alternate between local computations and communication with a coordinating server. Communication, which can be slow and costly, is the main bottleneck in this setting. To reduce this burden and therefore accelerate distributed gradient descent, two strategies are popular: (1) communicate less frequently; that is, perform several iterations of local computations between the communication rounds; and (2) communicate compressed information instead of full-dimensional vectors. We propose CompressedScaffnew, the first algorithm for distributed optimization that jointly harnesses these two strategies and converges linearly to an exact solution in the strongly convex setting, with a doubly accelerated rate: it benefits from the two acceleration mechanisms provided by local training and compression, namely a better dependency on the condition number of the functions and on the dimension of the model, respectively.
No takes yet. Share an insight, caveat, or question.
Condat et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: