Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Augmented Lagrangian methods for infeasible convex optimization problems and diverging proximal-point algorithms

Roland Andrews, Justin Carpentier, Adrien Taylor

Abstract

This work investigates the convergence behavior of augmented Lagrangian methods (ALMs) when applied to convex optimization problems that may be infeasible. ALMs are a popular class of algorithms for solving constrained optimization problems. We demonstrate that, under mild assumptions, the sequences of iterates generated by ALMs converge to solutions of the ``closest feasible problem''. We establish progressively stronger convergence results, ranging from basic sequence convergence to more precise convergence rates, under a hierarchy of assumptions. This study leverages the classical relationship between ALMs and the proximal-point algorithm applied to the dual problem. A key technical contribution is a set of concise results on the behavior of the proximal-point algorithm when applied to functions that may lack minimizers. These results pertain to its convergence in terms of its subgradients and of the values of the convex conjugate.

Augmented Lagrangian methods for infeasible convex optimization problems and diverging proximal-point algorithms

Abstract

This work investigates the convergence behavior of augmented Lagrangian methods (ALMs) when applied to convex optimization problems that may be infeasible. ALMs are a popular class of algorithms for solving constrained optimization problems. We demonstrate that, under mild assumptions, the sequences of iterates generated by ALMs converge to solutions of the ``closest feasible problem''. We establish progressively stronger convergence results, ranging from basic sequence convergence to more precise convergence rates, under a hierarchy of assumptions. This study leverages the classical relationship between ALMs and the proximal-point algorithm applied to the dual problem. A key technical contribution is a set of concise results on the behavior of the proximal-point algorithm when applied to functions that may lack minimizers. These results pertain to its convergence in terms of its subgradients and of the values of the convex conjugate.

Paper Structure

This paper contains 27 sections, 18 theorems, 90 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Infeasibility detection.
  4. The augmented Lagrangian method as a penalty method.
  5. Convergence of the augmented Lagrangian method on infeasible problems.
  6. Infeasible and inexact proximal point algorithms.
  7. Value function.
  8. Contributions
  9. Notations and Preliminaries
  10. Lagrangian and dual function.
  11. The value function.
  12. Inexact proximal point algorithm.
  13. Assumptions on the errors and step sizes.
  14. Reformulation of IALM as an inexact proximal point algorithm
  15. Inexact Proximal Point Algorithm
  16. ...and 12 more sections

Key Result

Proposition 1.3

rockafellar1976augmented Consider the convex optimization problem eq:convex_min_problem. For any sequence $(\lambda_\star^{k+1}, s_\star^{k+1}, \lambda^{k+1} , s^{k+1})_{k\in\mathbb{N}}$ generated by IALM (algo:IALM) on eq:convex_min_problem with positive penalty parameters $(\gamma_k)_{k\in\mathbb{

Figures (2)

  • Figure 1: Informal diagram summarizing the chain of implications between properties shown in \ref{['sec:convergence_ialm']}. Arrows represent sufficient conditions. Plus signs represent the logical AND operator.
  • Figure 2: Contour plot of the value function $\nu(s_1, s_2)$ for the QCQP example. $\nu$ is nondifferentiable at $\overline{s}$ when $(\alpha, \beta) = (- 1, 1)$ and subdifferentiable at $\overline{s}$ when $(\alpha, \beta) = (1, 1)$.

Theorems & Definitions (42)

  • Example 1.1: ALM for inequality-constrained convex optimization
  • Proposition 1.3
  • proof
  • Lemma 1.4
  • proof
  • Proposition 1.5
  • proof
  • Lemma 2.1
  • proof
  • Proposition 2.2
  • ...and 32 more