Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints

Lahcen El Bourkhissi, Ion Necoara, Panagiotis Patrinos, Quoc Tran-Dinh

TL;DR

This work develops a Linearized Perturbed Augmented Lagrangian method (LIPAL) for general nonsmooth, nonconvex optimization with nonlinear equality constraints. By perturbing the dual variables and performing prox-linear linearization inside a perturbed augmented Lagrangian, it yields convex subproblems and a tractable update rule, with global convergence to an $\epsilon$-first-order solution at a rate of $\mathcal{O}(\epsilon^{-3})$ evaluations. Under Kurdyka-Łojasiewicz structure, it derives improved local convergence rates, including linear and sublinear regimes, when the Lyapunov function satisfies the KL property. The paper also introduces a new constraint qualification that ensures bounded dual iterates, discusses regularity implications, and validates the approach via numerical experiments on large-scale clustering problems, illustrating practical efficiency and scalability.

Abstract

This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local smoothness. To tackle this challenging class of problems, we propose a novel linearized perturbed augmented Lagrangian method. This method incorporates a perturbation in the augmented Lagrangian function by scaling the dual variable with a sub-unitary parameter. Furthermore, we linearize the smooth components of the objective and the constraints within the perturbed Lagrangian function at the current iterate, while preserving the nonsmooth components. This approach, inspired by prox-linear (or Gauss-Newton) methods, results in a convex subproblem that is typically easy to solve. The solution of this subproblem then serves as the next primal iterate, followed by a perturbed ascent step to update the dual variables. Under a newly introduced constraint qualification condition, we establish the boundedness of the dual iterates. We derive convergence guarantees for the primal iterates, proving convergence to an $ε$-first-order optimal solution within $\mathcal{O}(ε^{-3})$ evaluations of the problem's functions and their first derivatives. Moreover, when the problem exhibits for example a semialgebraic property, we derive improved local convergence results. Finally, we validate the theoretical findings and assess the practical performance of our proposed algorithm through numerical comparisons with existing state-of-the-art methods.

Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints

TL;DR

This work develops a Linearized Perturbed Augmented Lagrangian method (LIPAL) for general nonsmooth, nonconvex optimization with nonlinear equality constraints. By perturbing the dual variables and performing prox-linear linearization inside a perturbed augmented Lagrangian, it yields convex subproblems and a tractable update rule, with global convergence to an -first-order solution at a rate of evaluations. Under Kurdyka-Łojasiewicz structure, it derives improved local convergence rates, including linear and sublinear regimes, when the Lyapunov function satisfies the KL property. The paper also introduces a new constraint qualification that ensures bounded dual iterates, discusses regularity implications, and validates the approach via numerical experiments on large-scale clustering problems, illustrating practical efficiency and scalability.

Abstract

This paper addresses a class of general nonsmooth and nonconvex composite optimization problems subject to nonlinear equality constraints. We assume that a part of the objective function and the functional constraints exhibit local smoothness. To tackle this challenging class of problems, we propose a novel linearized perturbed augmented Lagrangian method. This method incorporates a perturbation in the augmented Lagrangian function by scaling the dual variable with a sub-unitary parameter. Furthermore, we linearize the smooth components of the objective and the constraints within the perturbed Lagrangian function at the current iterate, while preserving the nonsmooth components. This approach, inspired by prox-linear (or Gauss-Newton) methods, results in a convex subproblem that is typically easy to solve. The solution of this subproblem then serves as the next primal iterate, followed by a perturbed ascent step to update the dual variables. Under a newly introduced constraint qualification condition, we establish the boundedness of the dual iterates. We derive convergence guarantees for the primal iterates, proving convergence to an -first-order optimal solution within evaluations of the problem's functions and their first derivatives. Moreover, when the problem exhibits for example a semialgebraic property, we derive improved local convergence results. Finally, we validate the theoretical findings and assess the practical performance of our proposed algorithm through numerical comparisons with existing state-of-the-art methods.

Paper Structure

This paper contains 13 sections, 16 theorems, 176 equations, 1 figure, 1 table, 2 algorithms.

Table of Contents

  1. Introduction.
  2. Problem Formulation and Model Assumptions.
  3. A Linearized Perturbed Augmented Lagrangian Method.
  4. Convergence Analysis.
  5. Technical lemmas
  6. Global convergence.
  7. Improved rates under KL condition.
  8. Selection of the penalty parameter $\rho$.
  9. Discussion on Regularity Conditions.
  10. Special cases of the regularity condition.
  11. Algorithmic perspective.
  12. Numerical experiments.
  13. Conclusions

Key Result

Lemma 1

If Assumption as:Assum holds on a compact set $\mathcal{S}$, then for any compact set $\mathcal{Y}\subset\mathbb{R}^m$ there exists $L^{\tau}_{\rho}>0$ such that: where $L^{\tau}_{\rho} \triangleq \sup_{(x,y)\in\mathcal{S}\times\mathcal{Y}}\left\{L_f+L_F\Vert \tau y^0 + (1-\tau)y +\rho F(x)\Vert +\rho M_F^2\right\}$.

Figures (1)

  • Figure 1: Clustering of $m=150$ random generated data of dimension $d=2$ using LIPAL, ALMS and SDPNAL+.

Theorems & Definitions (24)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Lemma 1: Smoothness of $\psi^{\tau}_{\rho}$
  • Lemma 2
  • Remark 1
  • Lemma 3: Bound for $\Vert y^{k}\Vert$
  • Lemma 4: Bound for $\Vert \Delta y^{k}\Vert$
  • Lemma 5: Descent property of $\mathcal{L}^{\tau}_{\rho}$
  • ...and 14 more