Ensuring the privacy of local datasets has emerged as an important concern in decentralized learning. However, the inherent privacy-utility tradeoff remains a fundamental challenge for privacy preserving decentralized algorithms. To address this issue, we introduce Positive-Incentive Noise Generator (PING), a novel mechanism designed to eliminate negative impact of privacy noise on convergence while defending against powerful colluding inference attacks. PING leverages network topologies and lightweight encryption-decryption operations to generate correlated noise. Building upon PING, we propose PP-DPIN, a privacy preserving stochastic algorithm tailored for decentralized learning. By integrating differential privacy and differential information entropy, we provide a comprehensive privacy quantification for PP-DPIN, with at least half nodes achieving arbitrarily strong privacy guarantees. Furthermore, convergence rate of PP-DPIN is established under stochastic convex and nonconvex settings, which characterizes the impact of privacy noise and demonstrates the linear speedup relative to the network size. Experiments on computer vision tasks validate PP-DPIN's superior performance and robustness against attacks compared to state-of-the-art methods.
Wang et al. (Thu,) studied this question.