Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Linear Convergence Result for the Jacobi-Proximal Alternating Direction Method of Multipliers

Hyelin Choi, Woocheol Choi

TL;DR

The paper proves global linear convergence of the Jacobi-Proximal ADMM for multi-block convex programs with linear constraints when objective components are strongly convex with Lipschitz gradients. It establishes a contraction of a Lyapunov-like function under explicit parameter conditions that tie proximal terms, damping, and penalty parameters to a linear rate. Theoretical results are supported by numerical experiments on LCQP and resource-allocation problems, illustrating practical effectiveness and parameter sensitivity. This work provides the first finite-rate linear guarantee for this parallel ADMM variant and demonstrates its viability for large-scale, multi-block optimization tasks.

Abstract

In this paper, we analyze the convergence rate of the Jacobi-Proximal Alternating Direction Method of Multipliers (ADMM) initially introduced by Deng et al. for the block-structured optimization problem with linear constraint. The algorithm is well-suited for parallel implementation and widely used for large-scale multi-block optimization problems. While the o(1/k) convergence of the Jacobi-Proximal ADMM for the case $N \geq 3$ has been well-established in the previous work, to the best of our knowledge, its linear convergence for $N \geq 3$ remains unproven. We establish the linear convergence of the algorithm when the cost functions are strongly convex and smooth. Numerical experiments are presented supporting the convergence result.

A Linear Convergence Result for the Jacobi-Proximal Alternating Direction Method of Multipliers

TL;DR

The paper proves global linear convergence of the Jacobi-Proximal ADMM for multi-block convex programs with linear constraints when objective components are strongly convex with Lipschitz gradients. It establishes a contraction of a Lyapunov-like function under explicit parameter conditions that tie proximal terms, damping, and penalty parameters to a linear rate. Theoretical results are supported by numerical experiments on LCQP and resource-allocation problems, illustrating practical effectiveness and parameter sensitivity. This work provides the first finite-rate linear guarantee for this parallel ADMM variant and demonstrates its viability for large-scale, multi-block optimization tasks.

Abstract

In this paper, we analyze the convergence rate of the Jacobi-Proximal Alternating Direction Method of Multipliers (ADMM) initially introduced by Deng et al. for the block-structured optimization problem with linear constraint. The algorithm is well-suited for parallel implementation and widely used for large-scale multi-block optimization problems. While the o(1/k) convergence of the Jacobi-Proximal ADMM for the case has been well-established in the previous work, to the best of our knowledge, its linear convergence for remains unproven. We establish the linear convergence of the algorithm when the cost functions are strongly convex and smooth. Numerical experiments are presented supporting the convergence result.

Paper Structure

This paper contains 10 sections, 7 theorems, 62 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Contribution
  4. Organization
  5. Statement of the main result
  6. Proof of the main result
  7. Numerical experiments
  8. The LCQP Model
  9. The Optimal Resource Allocation Problem
  10. Conclusion

Key Result

Theorem 2.6

Suppose Assumptions ass-1-ass-5 hold and assume that $P_i$ is a positive semi-definite matrix for each $1\leq i \leq N$. Choose any $s >0$ such that and suppose that $0<\gamma<2$, $\rho >0$ are chosen so that there exist values $\xi_i > 0$ such that Define $\sigma = \max \left\{ {1-2\gamma\rho sc_A^2, \mu_s}\right\} \in (0,1)$ for any $c_A \in (0, \frac{1}{\sqrt{2\gamma \rho s}})$, and $\mu_s \i

Figures (8)

  • Figure 1: Experimental results on LCQP model for $N=3$ ($m=100$, $n=40$), with fixed $\gamma$ and varying $\rho$.
  • Figure 2: Experimental results on LCQP model for $N=3$ ($m=100$, $n=40$), with fixed $\rho$ and varying $\gamma$.
  • Figure 3: Experimental results on LCQP model for $N=10$ ($m=100$, $n=60$), with fixed $\gamma$ and varying $\rho$.
  • Figure 4: Experimental results on LCQP model for $N=10$ ($m=100$, $n=60$), with fixed $\rho$ and varying $\gamma$.
  • Figure 5: Experimental results on the optimal resource allocation model for $N=6$, with fixed $\gamma$ and varying $\rho$.
  • ...and 3 more figures

Theorems & Definitions (16)

  • Theorem 2.6
  • Remark 2.7
  • Lemma 3.1
  • proof
  • Remark 3.2
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • Proposition 3.5
  • ...and 6 more