An accelerated semi-proximal ADMM with applications to multi-block sparse optimization problems
Peng Liu, Liang Chen, Minru Bai
TL;DR
The paper addresses the acceleration of semi-proximal ADMM for two-block and multi-block convex optimization, introducing AsPADMM with extrapolation and increasing penalties to achieve a non-ergodic $O(1/K)$ rate (versus $O(1/\sqrt{K})$ for standard sPADMM). By integrating with sGS decomposition, it extends the method to multi-block settings and preserves the accelerated rate. The authors provide convergence proofs and define proximal variants to ensure well-defined subproblems, then demonstrate practical benefits in robust low-rank tensor completion, mixed sparse optimization, and Lasso problems. This work offers a robust, fast-converging framework for large-scale multi-block convex programs with important implications for image processing and sparse/low-rank learning tasks.
Abstract
As an extension of the alternating direction method of multipliers (ADMM), the semi-proximal ADMM (sPADMM) has been widely used in various fields due to its flexibility and robustness. In this paper, we first show that the two-block sPADMM algorithm can achieve an $O(1/\sqrt{K})$ non-ergodic convergence rate. Then we propose an accelerated sPADMM (AsPADMM) algorithm by introducing extrapolation techniques and incrementing penalty parameters. The proposed AsPADMM algorithm is proven to converge globally to an optimal solution with a non-ergodic convergence rate of $O(1/K)$. Furthermore, the AsPADMM can be extended and combined with the symmetric Gauss-Seidel decomposition to achieve an accelerated ADMM for multi-block problems. Finally, we apply the proposed AsPADMM to solving the multi-block subproblems in difference-of-convex algorithms for robust low-rank tensor completion problems and mixed sparse optimization problems. The numerical results suggest that the acceleration techniques bring about a notable improvement in the convergence speed.