Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of a Relative-type Inexact Proximal ALM for Convex Nonlinear Programming

Lei Yang, Jiayi Zhu, Ling Liang, Kim-Chuan Toh

TL;DR

The paper addresses convex nonlinear programming with linear equality and convex nonlinear inequality constraints by proposing a relative-type inexact proximal augmented Lagrangian method (ripALM). It leverages a proximal augmentation to enforce strong convexity in subproblems, uses a relative-type inexact stopping criterion with a single tolerance $\rho$, and updates primal/dual variables without a correction step. The authors prove global convergence under mild assumptions, establish an asymptotic (super)linear convergence rate under an error-bound condition, and derive a global ergodic convergence rate that can be tuned via the penalty sequence $\{\sigma_k\}$. These results provide robust guarantees and practical guidance for applying ripALM to large-scale convex optimization, balancing accuracy, computational cost, and inexactness in subproblem minimization.

Abstract

This article investigates the convergence properties of a relative-type inexact proximal augmented Lagrangian method (ripALM) for convex nonlinear programming, a fundamental class of optimization problems with broad applications in science and engineering. Inexact proximal augmented Lagrangian methods have proven to be highly effective for solving such problems, owing to their attractive theoretical properties and strong practical performance. However, the convergence behavior of the relative-type inexact variant remains insufficiently understood. This work aims to reduce this gap by rigorously establishing the global convergence of the sequence generated by ripALM and proving its asymptotic (super)linear convergence rate under standard assumptions. In addition, we derive the global ergodic convergence rate with respect to both the primal feasibility violation and the primal objective residual, thereby offering a more comprehensive characterization of the overall performance of ripALM.

Convergence of a Relative-type Inexact Proximal ALM for Convex Nonlinear Programming

TL;DR

The paper addresses convex nonlinear programming with linear equality and convex nonlinear inequality constraints by proposing a relative-type inexact proximal augmented Lagrangian method (ripALM). It leverages a proximal augmentation to enforce strong convexity in subproblems, uses a relative-type inexact stopping criterion with a single tolerance , and updates primal/dual variables without a correction step. The authors prove global convergence under mild assumptions, establish an asymptotic (super)linear convergence rate under an error-bound condition, and derive a global ergodic convergence rate that can be tuned via the penalty sequence . These results provide robust guarantees and practical guidance for applying ripALM to large-scale convex optimization, balancing accuracy, computational cost, and inexactness in subproblem minimization.

Abstract

This article investigates the convergence properties of a relative-type inexact proximal augmented Lagrangian method (ripALM) for convex nonlinear programming, a fundamental class of optimization problems with broad applications in science and engineering. Inexact proximal augmented Lagrangian methods have proven to be highly effective for solving such problems, owing to their attractive theoretical properties and strong practical performance. However, the convergence behavior of the relative-type inexact variant remains insufficiently understood. This work aims to reduce this gap by rigorously establishing the global convergence of the sequence generated by ripALM and proving its asymptotic (super)linear convergence rate under standard assumptions. In addition, we derive the global ergodic convergence rate with respect to both the primal feasibility violation and the primal objective residual, thereby offering a more comprehensive characterization of the overall performance of ripALM.

Paper Structure

This paper contains 11 sections, 5 theorems, 100 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. A relative-type inexact proximal ALM
  3. Convergence analysis
  4. Global convergence
  5. Asymptotic (super)linear convergence rate
  6. Global ergodic convergence rate
  7. Conclusion
  8. A technical lemma
  9. Proof of Theorem \ref{['thm:convergence-gen']}
  10. Proof of Theorem \ref{['thm:Q-linear-rate-gen']}
  11. Proof of Theorem \ref{['thm:ergo-rate']}

Key Result

Theorem 3.1

Let the functions $F$, $G$ and $\ell$ be defined as in gen-paraobjpri, gen-paraobjdual and gen-para-lag, respectively. Let $\rho\in[0,1)$, $\{\sigma_{k}\}$ be a positive sequence satisfying that $\sigma_k\geq\sigma_{\min}>0$ for all $k\geq0$, and $\{\tau_{k}\}$ be a positive sequence satisfying that Let $\{\bm{x}^{k}\}$, $\{\Delta^{k}\}$, $\{\bm{w}^{k}\}\subset\mathbb{R}^{N}$ and $\{\bm{y}^{k}:=(\

Theorems & Definitions (12)

  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • proof
  • Corollary 3.1
  • Lemma A.1
  • proof
  • proof
  • ...and 2 more