Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptive Proximal ADMM for Nonconvex Linearly Constrained Composite Programs

Leandro Farias Maia, David H. Gutman, Renato D. C. Monteiro, Gilson N. Silva

TL;DR

This work develops Adapt-ADMM, a parameter-free, adaptive proximal ADMM for nonconvex linearly constrained composite programs with a weakly convex smooth part and block-separable convex nonsmooth part. The method employs adaptive inner solvers (ABIPP) and adaptive prox stepsizes, enabling inexact block updates and efficient dual/penalty updating without knowledge of problem weak convexity constants. Theoretical contributions include convergence guarantees to a $(\rho,\eta)$-stationary point with complexity ${\cal O}\left(B\max\{\rho^{-3},\eta^{-3}\}\right)$ iterations and improved dependence on the number of blocks relative to prior proximal-ADMM results, all without assuming a last-block condition. Numerical experiments across three nonconvex, block-structured problems demonstrate significant computational benefits and robustness of Adapt-ADMM compared to fixed-parameter variants.

Abstract

This paper develops an adaptive proximal alternating direction method of multipliers (ADMM) for solving linearly constrained, composite optimization problems under the assumption that the smooth component of the objective is weakly convex, while the non-smooth component is convex and block-separable. The proposed method is adaptive to all problem parameters, including smoothness and weak convexity constants, and allows each of its block proximal subproblems to be inexactly solved. Each iteration of our adaptive proximal ADMM consists of two steps: the sequential solution of each block proximal subproblem; and adaptive tests to decide whether to perform a full Lagrange multiplier and/or penalty parameter update(s). Without any rank assumptions on the constraint matrices, it is shown that the adaptive proximal ADMM obtains an approximate first-order stationary point of the constrained problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM's. The three proof-of-concept numerical experiments that conclude the paper suggest our adaptive proximal ADMM enjoys significant computational benefits.

An Adaptive Proximal ADMM for Nonconvex Linearly Constrained Composite Programs

TL;DR

This work develops Adapt-ADMM, a parameter-free, adaptive proximal ADMM for nonconvex linearly constrained composite programs with a weakly convex smooth part and block-separable convex nonsmooth part. The method employs adaptive inner solvers (ABIPP) and adaptive prox stepsizes, enabling inexact block updates and efficient dual/penalty updating without knowledge of problem weak convexity constants. Theoretical contributions include convergence guarantees to a -stationary point with complexity iterations and improved dependence on the number of blocks relative to prior proximal-ADMM results, all without assuming a last-block condition. Numerical experiments across three nonconvex, block-structured problems demonstrate significant computational benefits and robustness of Adapt-ADMM compared to fixed-parameter variants.

Abstract

This paper develops an adaptive proximal alternating direction method of multipliers (ADMM) for solving linearly constrained, composite optimization problems under the assumption that the smooth component of the objective is weakly convex, while the non-smooth component is convex and block-separable. The proposed method is adaptive to all problem parameters, including smoothness and weak convexity constants, and allows each of its block proximal subproblems to be inexactly solved. Each iteration of our adaptive proximal ADMM consists of two steps: the sequential solution of each block proximal subproblem; and adaptive tests to decide whether to perform a full Lagrange multiplier and/or penalty parameter update(s). Without any rank assumptions on the constraint matrices, it is shown that the adaptive proximal ADMM obtains an approximate first-order stationary point of the constrained problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM's. The three proof-of-concept numerical experiments that conclude the paper suggest our adaptive proximal ADMM enjoys significant computational benefits.
Paper Structure (24 sections, 22 theorems, 119 equations, 3 tables, 5 algorithms)

This paper contains 24 sections, 22 theorems, 119 equations, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Structure of the Paper
  3. Notation, Definitions, and Basic Facts
  4. Assumptions and an Inexact Solution Concept
  5. Assumptions for Problem \ref{['initial.problem']}-\ref{['eq:block_structure']}
  6. An Inexact Solution Concept for \ref{['eq:x_update']}
  7. A Fixed Penalty ADMM
  8. The BIPP Subroutine
  9. Description of FP-ADMM
  10. The Proof of FP-ADMM's Complexity Theorem (Theorem \ref{['thm:static.complexity']})
  11. A Variable Penalty ADMM
  12. An adaptive prox stepsize VP-ADMM
  13. A-BIPP: A variable prox stepsize version of BIPP
  14. An adaptive prox stepsize VP-ADMM
  15. Further implementation issues of a A-BIPP loop
  16. ...and 9 more sections

Key Result

Proposition 3.1

Assume that $(z^+,v^+,\delta_+)={\textproc{{BIPP}}}(z,p,{\lambda}, c)$ for some $(z,p,{\lambda}, c) \in {\cal H} \times A(\mathbb{R}^n) \times \mathbb{R}_{++}^B\times \mathbb{R}_{++}$ and the prox stepsize ${\lambda} \in \mathbb{R}^B_{++}$ input to BIPP is chosen as Then, the following statements hold:

Theorems & Definitions (36)

  • Definition 2.1
  • Proposition 3.1
  • Theorem 3.2: FP-ADMM Complexity
  • Lemma 4.1
  • proof
  • Lemma 4.2
  • proof
  • Lemma 4.3
  • proof
  • Proposition 4.4
  • ...and 26 more