The inexact power augmented Lagrangian method for constrained nonconvex optimization

TL;DR

This work introduces an inexact power augmented Lagrangian method for nonconvex problems with nonlinear equality constraints by using a power-type augmenting term φ(A(x))=(1/(ν+1))||A(x)||^{ν+1} with ν∈(0,1]. The authors develop an outer-inexact ALM framework and an inner proximal-point solver for Hölder-smooth subproblems, establishing rates where the dual progress slows while primal feasibility speeds up as ν decreases. They provide comprehensive complexity bounds, showing that ν<1 yields faster constraint satisfaction at the cost of slower objective decrease, and that linear constraint cases yield further improvements; ν=1 recovers the best-known ε^{-3}–ε^{-4} rates under standard assumptions. The paper also includes a robust analysis that does not require compact iterates and accommodates a generic convex term g, with numerical experiments in clustering and quadratic programs confirming the practical benefits of unconventional penalties. Overall, the results highlight a principled trade-off between primal and dual complexity and demonstrate the practical impact of using higher-order penalty terms in constrained nonconvex optimization.

Abstract

This work introduces an unconventional inexact augmented Lagrangian method where the augmenting term is a Euclidean norm raised to a power between one and two. The proposed algorithm is applicable to a broad class of constrained nonconvex minimization problems that involve nonlinear equality constraints. In a first part of this work, we conduct a full complexity analysis of the method under a mild regularity condition, leveraging an accelerated first-order algorithm for solving the Hölder-smooth subproblems. Interestingly, this worst-case result indicates that using lower powers for the augmenting term leads to faster constraint satisfaction, albeit with a slower decrease of the dual residual. Notably, our analysis does not assume boundedness of the iterates. Thereafter, we present an inexact proximal point method for solving the weakly-convex and Hölder-smooth subproblems, and demonstrate that the combined scheme attains an improved rate that reduces to the best-known convergence rate whenever the augmenting term is a classical squared Euclidean norm. Different augmenting terms, involving a lower power, further improve the primal complexity at the cost of the dual complexity. Finally, numerical experiments validate the practical performance of unconventional augmenting terms.

Figures (4)

• Figure 1: Comparison of power ALM with various powers $\nu$ on solving the clustering problem with Fashion-MNIST data xiao_fashion-mnist_2017. The case $\nu = 1$ corresponds to iALM from sahin_inexact_2019.
• Figure 2: Comparison of power ALM with various powers $\nu$ on solving the clustering problem with MNIST data deng_mnist_2012. The case $\nu = 1$ corresponds to iALM from sahin_inexact_2019.
• Figure 3: Power ALM for various powers $\nu$ on solving $100$ random QPs of size $n = 100, m = 20$.
• Figure 4: Comparison of the proposed power ALM with various powers $\nu$ on solving a representative GEVP with $n = 500$. The case $\nu = 1$ corresponds to the iALM from sahin_inexact_2019.

Theorems & Definitions (31)

• Definition 1: $(\varepsilon_\varphi, \varepsilon_A)$-stationary points
• Lemma 1
• Lemma 2
• Theorem 1: Outer complexity
• Lemma 3: Augmented Lagrangian smoothness
• Lemma 4: Inner complexity
• Theorem 2: Total complexity
• Lemma 5
• Theorem 3
• Theorem 4