What type of study is this?

This is a Quantitative Study study.

synapse

⌘+K

synapse

⌘+K

October 20, 2025Open Access

High Probability Convergence of Distributed Clipped Stochastic Gradient Descent with Heavy-tailed Noise

Key Points

The proposed algorithm converges with high probability in the presence of heavy-tailed noise, addressing a gap in current research.
Under mild connectivity assumptions, the algorithm utilizes a clipping operator to ensure convergence.
A simulation showcases the algorithm's effectiveness compared to existing methodologies in stochastic gradient descent settings.
Traditional distributed gradient algorithms typically assume light-tailed noise, making this study's focus significant for practical applications.

Abstract

In this paper, the problem of distributed optimization is studied via a network of agents. Each agent only has access to a noisy gradient of its own objective function, and can communicate with its neighbors via a network. To handle this problem, a distributed clipped stochastic gradient descent algorithm is proposed, and the high probability convergence of the algorithm is studied. Existing works on distributed algorithms involving stochastic gradients only consider the light-tailed noises. Different from them, we study the case with heavy-tailed settings. Under mild assumptions on the graph connectivity, we prove that the algorithm converges in high probability under a certain clipping operator. Finally, a simulation is provided to demonstrate the effectiveness of our theoretical results

Read Full Paperexternally

Mark Helpful

Bookmark

Relay

View Full Paper