Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local Convergence Analysis of ADMM for Nonconvex Composite Optimization

Xiyuan Xie, Lihua Yang, Qia li

Abstract

In this paper, we study the local convergence of the standard ADMM scheme for a class of nonconvex composite problems arising from modern imaging and machine learning models. This problem is constrained by a closed convex set, while its objective is the sum of a continuously differentiable (possibly nonconvex) smooth term and a polyhedral convex nonsmooth term composed with a linear mapping. Our analysis is mainly motivated by the recent works of Rockafellar [29,30]. We begin with an elementary proof of a key local strong convexity property of the Moreau envelope of polyhedral convex functions. Building on this property, we show that the strong variational sufficiency condition holds for the considered problem under appropriate assumptions. Using the strong variational sufficiency condition, we further derive a descent inequality for the ADMM iterates, in a form analogous to the classical descent analysis of ADMM for convex problems. As a consequence, for a suitable choice of the penalty parameter, we establish local convergence of the ADMM scheme to a primal-dual solution, and a local linear convergence rate for the case where the constraint set is polyhedral convex. Finally, we present three analytic examples to illustrate the applicability of our local convergence result and the necessity of the local assumptions.

Local Convergence Analysis of ADMM for Nonconvex Composite Optimization

Abstract

In this paper, we study the local convergence of the standard ADMM scheme for a class of nonconvex composite problems arising from modern imaging and machine learning models. This problem is constrained by a closed convex set, while its objective is the sum of a continuously differentiable (possibly nonconvex) smooth term and a polyhedral convex nonsmooth term composed with a linear mapping. Our analysis is mainly motivated by the recent works of Rockafellar [29,30]. We begin with an elementary proof of a key local strong convexity property of the Moreau envelope of polyhedral convex functions. Building on this property, we show that the strong variational sufficiency condition holds for the considered problem under appropriate assumptions. Using the strong variational sufficiency condition, we further derive a descent inequality for the ADMM iterates, in a form analogous to the classical descent analysis of ADMM for convex problems. As a consequence, for a suitable choice of the penalty parameter, we establish local convergence of the ADMM scheme to a primal-dual solution, and a local linear convergence rate for the case where the constraint set is polyhedral convex. Finally, we present three analytic examples to illustrate the applicability of our local convergence result and the necessity of the local assumptions.
Paper Structure (6 sections, 12 theorems, 95 equations, 1 figure)

This paper contains 6 sections, 12 theorems, 95 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Notation and Preliminaries
  3. Strong Variational Sufficiency for Problem \ref{['prob:com']}
  4. Local Convergence Analysis
  5. Examples
  6. Conclusion

Key Result

Proposition 1

Let $\mu>\nolinebreak 0$ and $\varphi:\mathbb{R}^m\to\overline{\mathbb{R}}$ be a proper closed convex function, then for every $w\in\mathbb{R}^m$, there exists a unique pair $(u_w,\lambda^{w})$ satisfying where $u_w=\mathrm{prox}_{\mu \varphi}(w)$ and $\mu\lambda^w=\mathrm{prox}_{(\mu \varphi)^*}(w)$.

Figures (1)

  • Figure 1: $s_k:=\|x^k-\bar{x}\|+\|y^k-A\bar{x}\|+\|\lambda^k-\bar{\lambda}\|$(in log scale)vs Iters

Theorems & Definitions (26)

  • Proposition 1: eckstein2015understanding
  • Proposition 2: zheng2014metric
  • Definition 1
  • Corollary 3
  • proof
  • Lemma 4
  • proof
  • Lemma 5
  • proof
  • Proposition 6
  • ...and 16 more