Inertial Relaxed Proximal Linearized ADMM for Nonconvex Optimization under Minimal Continuity Assumption

This paper proposes an Inertial Relaxed Proximal Linearized Alternating Direction Method of Multipliers (IRPL-ADMM) for solving general multi-block nonconvex composite optimization problems. Distinguishing itself from existing ADMM-style algorithms, our approach imposes a less stringent condition, specifically requiring continuity in only one block of the objective function. It incorporates an inertial strategy for primal variable updates, and a relaxed strategy for dual variable updates. The fundamental concept underlying our algorithm is based on novel regular penalty update rules, ensuring that the penalty increases but not excessively fast. We devise a novel potential function to facilitate our convergence analysis and extend our methods from deterministic optimization problems to finite-sum stochastic settings. We establish the iteration complexity for both scenarios for achieving an approximate stationary solution. Under the Kurdyka-Lojasiewicz (KL) inequality, we establish strong limit-point convergence results for the IRPL-ADMM algorithm. Finally, some experiments have been conducted on two machine learning tasks to show the effectiveness of our approaches.