Distributed random coordinate descent with gradient-aware sampling and sparse communication for large-scale sparse optimization

Large-scale sparse optimization with feature dimensions exceeding 10 5 and sparsity above 90% poses significant challenges for traditional gradient-based methods due to long training periods and heavy communication burdens. This paper proposes a distributed random coordinate descent algorithm integrating gradient-aware coordinate sampling and sparse communication compression. On the Criteo dataset (1.2M samples, 10 6 dimensions, 95% sparsity), the proposed method reduces the loss from 1.25 to 0.41 within the first 200 iterations, reaching stability at 0.349 after 900 iterations. Total training time under 32 nodes is 210 s, compared to 980 s for single-machine coordinate descent (4.7 × speedup). Communication load ranges from 0.9 MB (4 nodes) to 2.3 MB (32 nodes), significantly lower than distributed SGD (6.8 MB at 32 nodes). This work provides a scalable distributed optimization framework that maintains stable convergence while reducing communication overhead by approximately 70% compared to conventional distributed SGD in high-dimensional sparse scenarios.