Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A high-order augmented Lagrangian method with arbitrarily fast convergence

Young-Ju Lee, Jongho Park

TL;DR

A high-order version of the augmented Lagrangian method is proposed for solving convex optimization problems with linear constraints, which achieves arbitrarily fast convergence rates and applications to various problems arising in the sciences, including data fitting, flow in porous media, and scientific machine learning.

Abstract

We propose a high-order version of the augmented Lagrangian method for solving convex optimization problems with linear constraints, which achieves arbitrarily fast -- and even superlinear -- convergence rates. First, we analyze the convergence rates of the high-order proximal point method under certain uniform convexity assumptions on the energy functional. We then introduce the high-order augmented Lagrangian method and analyze its convergence by leveraging the convergence results of the high-order proximal point method. Finally, we present applications of the high-order augmented Lagrangian method to various problems arising in the sciences, including data fitting, flow in porous media, and scientific machine learning.

A high-order augmented Lagrangian method with arbitrarily fast convergence

TL;DR

A high-order version of the augmented Lagrangian method is proposed for solving convex optimization problems with linear constraints, which achieves arbitrarily fast convergence rates and applications to various problems arising in the sciences, including data fitting, flow in porous media, and scientific machine learning.

Abstract

We propose a high-order version of the augmented Lagrangian method for solving convex optimization problems with linear constraints, which achieves arbitrarily fast -- and even superlinear -- convergence rates. First, we analyze the convergence rates of the high-order proximal point method under certain uniform convexity assumptions on the energy functional. We then introduce the high-order augmented Lagrangian method and analyze its convergence by leveraging the convergence results of the high-order proximal point method. Finally, we present applications of the high-order augmented Lagrangian method to various problems arising in the sciences, including data fitting, flow in porous media, and scientific machine learning.
Paper Structure (12 sections, 9 theorems, 73 equations, 7 figures, 2 algorithms)

This paper contains 12 sections, 9 theorems, 73 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. High-order proximal point method
  4. High-order augmented Lagrangian method
  5. Computational aspects
  6. Numerical stability of dual subproblems
  7. Nearly semicoercive primal subproblems
  8. Applications
  9. ls data fitting under linear constraints
  10. Darcy--Forchheimer model
  11. Finite neuron method
  12. Conclusion

Key Result

Proposition 2.1

Let $F \colon V \to \mathbb{R}$ be a uniformly convex and $(q, L)$-weakly smooth functional for some $1 < q \leq 2$ and $L > 0$. Then its Legendre--Fenchel conjugate $F^*$ is $(q^*, L^{-(q^*-1)})$-uniformly convex with respect to the dual norm $\| \cdot \|_*$.

Figures (7)

  • Figure 1: Norm error $\| \lambda^{(n)} - \lambda \|$ of the high-order augmented Lagrangian method (\ref{['Alg:ALM']}) for solving the three applications discussed in \ref{['Sec:Applications']} ($r = 3$, $\epsilon = 10^{-2}$). The legends "unstable" and "stable" indicate that the dual updates are performed using \ref{['dual_update_unstable']} and \ref{['dual_update_stable']}, respectively.
  • Figure 1: Norm error $\| u^{(n)} - u \|$ of the high-order augmented Lagrangian method (\ref{['Alg:ALM']}) for solving the constrained $\ell^s$ location problem \ref{['location']} with $s = 3.0$.
  • Figure 2: Norm error $\| u^{(n)} - u \|$ of the high-order augmented Lagrangian method (\ref{['Alg:ALM']}) for solving the constrained $\ell^s$ location problem \ref{['location']} with $s = 1.5$.
  • Figure 3: Velocity $L^2$-norm error $\| \boldsymbol{u}_h^{(n)} - \boldsymbol{u}_h \|_{L^2 (\Omega)}$ of the high-order augmented Lagrangian method \ref{['Forchheimer_ALM']} for solving the Darcy--Forchheimer model \ref{['Forchheimer_opt']}.
  • Figure 4: Pressure $L^2$-norm error $\| p_h^{(n)} - p_h \|_{L^2 (\Omega)}$ of the high-order augmented Lagrangian method \ref{['Forchheimer_ALM']} for solving the Darcy--Forchheimer model \ref{['Forchheimer_opt']}.
  • ...and 2 more figures

Theorems & Definitions (20)

  • Proposition 2.1
  • Proof 1
  • Remark 2.2
  • Theorem 3.1
  • Proof 2
  • Theorem 3.2
  • Proof 3
  • Theorem 3.3
  • Proof 4
  • Theorem 4.1
  • ...and 10 more