Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Active set identification and rapid convergence for degenerate primal-dual problems

Mateo Díaz, Pedro Izquierdo Lehmann, Haihao Lu, Jinwen Yang

TL;DR

The paper addresses finite-time active-set identification for convex primal-dual problems solved by first-order methods, even in degenerate settings where strict complementarity fails. It introduces a meta-algorithm framework with a two-sequence scheme and metric-subregularity-based regularity, yielding nonasymptotic bounds for when the algorithm identifies the active set and then achieves linear convergence on a reduced, better-conditioned problem. The main contributions are explicit finite-time identification bounds depending on the local modulus $\alpha_L$ and a fast-linear convergence phase governed by $\alpha_{\mathcal{M}}$, applicable to PPM, PDHG, ADMM, and EGM. The results deliver practical, robust guarantees for two-stage convergence in a broad class of primal-dual algorithms, including linear and convex programs, and are corroborated by numerical experiments showing identification and rapid convergence even under degeneracy.

Abstract

Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at a sublinear rate. Then, after a certain point, the method identifies the set of active constraints and the convergence enters a faster local linear regime. Theory characterizing this phenomenon spans over three decades. However, most existing work only guarantees eventual identification of the active set and relies heavily on nondegeneracy conditions, such as strict complementarity, which often fail to hold in practice. We characterize mild conditions on the problem geometry and the algorithm under which this phenomenon provably occurs. Our guarantees are entirely nonasymptotic and, importantly, do not rely on strict complementarity. Our framework encompasses several widely-used algorithms, including the proximal point method, the primal-dual hybrid gradient method, the alternating direction method of multipliers, and the extragradient method.

Active set identification and rapid convergence for degenerate primal-dual problems

TL;DR

The paper addresses finite-time active-set identification for convex primal-dual problems solved by first-order methods, even in degenerate settings where strict complementarity fails. It introduces a meta-algorithm framework with a two-sequence scheme and metric-subregularity-based regularity, yielding nonasymptotic bounds for when the algorithm identifies the active set and then achieves linear convergence on a reduced, better-conditioned problem. The main contributions are explicit finite-time identification bounds depending on the local modulus and a fast-linear convergence phase governed by , applicable to PPM, PDHG, ADMM, and EGM. The results deliver practical, robust guarantees for two-stage convergence in a broad class of primal-dual algorithms, including linear and convex programs, and are corroborated by numerical experiments showing identification and rapid convergence even under degeneracy.

Abstract

Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at a sublinear rate. Then, after a certain point, the method identifies the set of active constraints and the convergence enters a faster local linear regime. Theory characterizing this phenomenon spans over three decades. However, most existing work only guarantees eventual identification of the active set and relies heavily on nondegeneracy conditions, such as strict complementarity, which often fail to hold in practice. We characterize mild conditions on the problem geometry and the algorithm under which this phenomenon provably occurs. Our guarantees are entirely nonasymptotic and, importantly, do not rely on strict complementarity. Our framework encompasses several widely-used algorithms, including the proximal point method, the primal-dual hybrid gradient method, the alternating direction method of multipliers, and the extragradient method.
Paper Structure (58 sections, 25 theorems, 182 equations, 8 figures)

This paper contains 58 sections, 25 theorems, 182 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Convex-concave primal-dual algorithms.
  3. Finite time identification.
  4. Metric subregularity and growth conditions.
  5. Preliminaries
  6. Setting and algorithms
  7. Problem class
  8. Algorithmic template
  9. Instantiations of the meta-algorithm
  10. Proximal Point Method (PPM).
  11. Primal Dual Hybrid Gradient (PDHG).
  12. Alternating Direction Method of Multipliers (ADMM).
  13. Extragradient Method (EGM).
  14. Guarantees
  15. Finite time identification
  16. ...and 43 more sections

Key Result

Proposition 3.1

Suppose Assumptions assumption:problem, assumption:metric-subregularity, and assumption:rapidLocalConvergence hold. Let $z^{k}$ be the $k$th iterate generated by update eq:algorithm. Then, Further, for any $k \in {\mathbf N}$ we have

Figures (8)

  • Figure 1: Distance to solution versus iteration count for several algorithms applied to a degenerate QP. The vertical dotted lines represent the last iteration at which the active set changed.
  • Figure 2: Illustration of the radius of active-set stability \ref{['eq:degeneracy-metric']}.
  • Figure 3: Example \ref{['ex:rotatedHouse']}. The left plot shows the feasible set and solutions $\mathcal{S}^\star.$ The right plot displays the regions where the metric subregularity moduli could land versus $c_1$ (we take $c_1 \leq c_2).$
  • Figure 4: Root-node relaxation of MIPLIB 2017 linear programs. Vertical dotted lines indicate active set identification. Figures \ref{['fig:LP2']} and \ref{['fig:LPdegen']} all methods converge to degenerate solutions. In \ref{['fig:LPdegen']}, ADMM identifies $\mathcal{M}$ and converges in just a couple of iterations.
  • Figure 5: Maros-Meszaros convex quadratic programs. Vertical dotted lines indicate active set identification. In \ref{['fig:QPdegen']}, the only algorithm that converged after $10^5$ iterations was ADMM; PDHG and EGM showed very slow progress.
  • ...and 3 more figures

Theorems & Definitions (43)

  • Proposition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 3.4
  • Proposition 3.5
  • Theorem 4.1
  • Theorem 4.2
  • Proposition 4.3
  • Proposition 4.4
  • Example 4.5
  • ...and 33 more