Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Smoothed Duality Gap as a Stopping Criterion

Iyad Walwil, Olivier Fercoq

Abstract

We optimize the running time of the primal-dual algorithms by optimizing their stopping criteria for solving convex optimization problems under affine equality constraints, which means terminating the algorithm earlier with fewer iterations. We study the relations between four stopping criteria and show under which conditions they are accurate to detect optimal solutions. The uncomputable one: ''Optimality gap and Feasibility error'', and the computable ones: the ''Karush-Kuhn-Tucker error'', the ''Projected Duality Gap'', and the ''Smoothed Duality Gap''. Assuming metric sub-regularity or quadratic error bound, we establish that all of the computable criteria provide practical upper bounds for the optimality gap, and approximate it effectively. Furthermore, we establish comparability between some of the computable criteria under certain conditions. Numerical experiments on basis pursuit, and quadratic programs with(out) non-negative weights corroborate these findings and show the superior stability of the smoothed duality gap over the rest.

The Smoothed Duality Gap as a Stopping Criterion

Abstract

We optimize the running time of the primal-dual algorithms by optimizing their stopping criteria for solving convex optimization problems under affine equality constraints, which means terminating the algorithm earlier with fewer iterations. We study the relations between four stopping criteria and show under which conditions they are accurate to detect optimal solutions. The uncomputable one: ''Optimality gap and Feasibility error'', and the computable ones: the ''Karush-Kuhn-Tucker error'', the ''Projected Duality Gap'', and the ''Smoothed Duality Gap''. Assuming metric sub-regularity or quadratic error bound, we establish that all of the computable criteria provide practical upper bounds for the optimality gap, and approximate it effectively. Furthermore, we establish comparability between some of the computable criteria under certain conditions. Numerical experiments on basis pursuit, and quadratic programs with(out) non-negative weights corroborate these findings and show the superior stability of the smoothed duality gap over the rest.
Paper Structure (9 sections, 20 theorems, 52 equations, 1 figure)

This paper contains 9 sections, 20 theorems, 52 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Our contributions and Paper organization
  3. Notations
  4. Background
  5. Optimality measures
  6. Projected Duality Gap
  7. Smoothed Duality Gap
  8. Optimality Gap Bounds
  9. Regularity assumptions

Key Result

proposition thmcounterproposition

For all vectors $\mathbf{u}$ and $\mathbf{v}$ of an inner product space, and for any scalar $\lambda$. The following inequality holds:

Figures (1)

  • Figure 1: Gradient descent is employed to address an unconstrained Least-Squares problem, utilizing two distinct stopping criteria aimed at achieving an $\varepsilon = 10^{-5}$ solution

Theorems & Definitions (38)

  • proposition thmcounterproposition: Young's inequality
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • proposition thmcounterproposition: Projection properties
  • definition thmcounterdefinition: Separable function
  • proposition thmcounterproposition: Properties of Separable function
  • definition thmcounterdefinition: Fenchel-Legendre Conjugate
  • proposition thmcounterproposition: Fenchel-Young's inequality
  • proposition thmcounterproposition
  • ...and 28 more