Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A projection-free dynamics for nonsmooth composite optimization

Wei Ni, Yangfan Qiu, Yanyan Xiao

TL;DR

This paper develops a projection-free, continuous-time primal-dual framework for nonsmooth composite optimization with general convex equality and inequality constraints by leveraging mirror descent and a proximal augmented Lagrangian (PAL). It proves the PAL is strongly convex in the primal variable and strongly concave in the dual block, enabling exponential convergence of the resulting dynamics (under LICQ and strong convexity assumptions) and extends convergence guarantees to general nonlinear convex inequalities without relying on IQC. A fully smooth, projection-free dynamics is designed by combining gradient descent on the primal variables with mirror ascent on the dual multipliers, avoiding nonsmooth differential inclusions. The approach is validated via a distributed optimization simulation on a Rosen–Suzuki-style problem, illustrating convergence of all nodes to the centralized optimum while satisfying constraints. Overall, the work broadens projection-free optimization to general convex constraints and provides rigorous exponential convergence guarantees with a fully smooth algorithmic structure.

Abstract

This paper proposes a projection-free primal-dual dynamics for the nonsmooth composite optimization problems with equality and inequality constraints. To deal with optimization constraints, this paper departs from the use of gradient projection method, but resorts to the idea of mirror descent to design a continuous-time smooth optimization dynamics which advantageously leads to easier convergence analysis and more efficient numerical simulation. Also, the strategy of proximal augmented Lagrangian (PAL$^†$) is extended to incorporate general convex equality-inequality constraints and the strong convexity-concavity of the primal-dual variables is achieved, ensuring exponential convergence of the resulting algorithm. Furthermore, the convergence result in this paper extends existing exponential convergence which either takes no account of constraints or considers only affine linear constraints, and it also enhances existing asymptotic convergence under convex constraints which unfortunately depends on the complex gradient projection scheme.

A projection-free dynamics for nonsmooth composite optimization

TL;DR

This paper develops a projection-free, continuous-time primal-dual framework for nonsmooth composite optimization with general convex equality and inequality constraints by leveraging mirror descent and a proximal augmented Lagrangian (PAL). It proves the PAL is strongly convex in the primal variable and strongly concave in the dual block, enabling exponential convergence of the resulting dynamics (under LICQ and strong convexity assumptions) and extends convergence guarantees to general nonlinear convex inequalities without relying on IQC. A fully smooth, projection-free dynamics is designed by combining gradient descent on the primal variables with mirror ascent on the dual multipliers, avoiding nonsmooth differential inclusions. The approach is validated via a distributed optimization simulation on a Rosen–Suzuki-style problem, illustrating convergence of all nodes to the centralized optimum while satisfying constraints. Overall, the work broadens projection-free optimization to general convex constraints and provides rigorous exponential convergence guarantees with a fully smooth algorithmic structure.

Abstract

This paper proposes a projection-free primal-dual dynamics for the nonsmooth composite optimization problems with equality and inequality constraints. To deal with optimization constraints, this paper departs from the use of gradient projection method, but resorts to the idea of mirror descent to design a continuous-time smooth optimization dynamics which advantageously leads to easier convergence analysis and more efficient numerical simulation. Also, the strategy of proximal augmented Lagrangian (PAL) is extended to incorporate general convex equality-inequality constraints and the strong convexity-concavity of the primal-dual variables is achieved, ensuring exponential convergence of the resulting algorithm. Furthermore, the convergence result in this paper extends existing exponential convergence which either takes no account of constraints or considers only affine linear constraints, and it also enhances existing asymptotic convergence under convex constraints which unfortunately depends on the complex gradient projection scheme.
Paper Structure (8 sections, 5 theorems, 37 equations, 2 figures)

This paper contains 8 sections, 5 theorems, 37 equations, 2 figures.

Table of Contents

  1. INTRODUCTION
  2. Preliminaries
  3. Problem formulation
  4. Optimization dynamics and convergence analysis
  5. Design of PAL
  6. Projection-free primal-dual optimization dynamics
  7. Simulation example
  8. Conclusions

Key Result

Lemma 4.1

Under Assumptions sconvex and lcontinuous, the PAL defined in augl is $\alpha$-strongly convex in $x$ and $\left(\frac{\mu\ell}{\mu+\ell}+2\mu\right)$-strongly concave in $(\lambda, { {\hbox{$\m@th\mkern1.2mu\mathchar'26$}}\lambda }, { {\hbox{$\m@th\mkern1.2mu\mathchar'26$}} {\hbox{$\m@th\mkern1.

Figures (2)

  • Figure 1: The topology of five agents described by a graph.
  • Figure 2: The time evolution of the states for 5 agents. (a) The first components of the five agents all converge to $0$; (b)The second components of the five agents all converge to $1$; (c) The third components of the five agents all converge to $2$; (d)The fourth components of the five agents all converge to $-1$. Therefore, each state of the 5 agents converges to the optimal solution $(0, 1, 2, -1)$.

Theorems & Definitions (9)

  • Lemma 4.1
  • Lemma 4.2
  • Remark 4.3
  • Lemma 4.4
  • Theorem 4.5
  • Remark 4.6
  • Remark 4.7
  • Theorem 4.8
  • Remark 4.9