An Introduction of Distributed Optimization Theory and Applications

This paper introduces distributed optimization theory, and its applications. Specifically, in this paper, we study detailed mathematical setups and mathematical proofs of the main theoretical works in distributed optimization. The theory of distributed optimization could be divided into two main different classes. The first one is developed in the "line search" manner such as gradient descent method, and the second one is developed from Lagrangian approach. Additionally, we present some important sub-algorithms in these two classes. We also introduce several domestic researchers who have been working in this field.