Smoothness of the Augmented Lagrangian Dual in Convex Optimization
Jingwang Li, Vincent Lau
TL;DR
This work tackles the convex optimization problem $\min f(x)$ subject to $A x = b$ with $f$ closed proper convex and analyzes the augmented Lagrangian dual $\phi_{\rho}(\lambda)$. The main result shows that $\phi_{\rho}$ is everywhere $\frac{1}{\rho}$-smooth and that for every dual variable $\lambda$, the inner minimization $\min_x \mathcal{L}_{\rho}(x, \lambda)$ has a solution, with $\nabla \phi_{\rho}(\lambda) = A x^+ - b$. A key technical contribution is proving the equivalence $M_{\rho(-\phi)} = -\phi_{\rho}$, coupling the dual to the Moreau envelope and enabling the smoothness analysis under the sole assumption that $f$ is closed proper convex. These results substantially relax prior regularity requirements and pave the way for rigorous convergence and complexity analyses of ALM variants in a broad class of problems.
Abstract
This paper investigates the general linearly constrained optimization problem: $\min_{x \in \R^d} f(x) \ \st \ A x = b$, where $f: \R^n \rightarrow \exs$ is a closed proper convex function, $A \in \R^{p \times d}$, and $b \in \R^p$. We establish the following results without requiring additional regularity conditions: (1) the augmented Lagrangian dual function $φ_ρ(λ) = \inf_x \cL_ρ(x, λ)$ is $\frac{1}ρ$-smooth everywhere; and (2) the solution to $\min_{x \in \R^d} \cL_ρ(x, λ)$ exists for any dual variable $λ\in \R^p$, where $ρ> 0$ is the augmented parameter and $\cL_ρ(x, λ) = f(x) + \dotprod{λ, A x - b} + \fracρ{2}\norm{A x - b}^2$ is the augmented Lagrangian. These findings significantly relax the strong assumptions commonly imposed in existing literature to guarantee similar properties.