Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parametrization and convergence of a primal-dual block-coordinate approach to linearly-constrained nonsmooth optimization

Olivier Bilenne

TL;DR

A randomized block-coordinate interpretation of the Chambolle-Pock primal-dual algorithm is studied, based on inexact proximal gradient steps, to derive tight sublinear convergence rates with and without duality assumptions and for both the convex and the strongly convex settings.

Abstract

This note is concerned with the problem of minimizing a separable, convex, composite (smooth and nonsmooth) function subject to linear constraints. We study a randomized block-coordinate interpretation of the Chambolle-Pock primal-dual algorithm, based on inexact proximal gradient steps. A specificity of the considered algorithm is its robustness, as it converges even in the absence of strong duality or when the linear program is inconsistent. Using matrix preconditiong, we derive tight sublinear convergence rates with and without duality assumptions and for both the convex and the strongly convex settings. Our developments are extensions and particularizations of original algorithms proposed by Malitsky (2019) and Luke and Malitsky (2018). Numerical experiments are provided for an optimal transport problem of service pricing.

Parametrization and convergence of a primal-dual block-coordinate approach to linearly-constrained nonsmooth optimization

TL;DR

A randomized block-coordinate interpretation of the Chambolle-Pock primal-dual algorithm is studied, based on inexact proximal gradient steps, to derive tight sublinear convergence rates with and without duality assumptions and for both the convex and the strongly convex settings.

Abstract

This note is concerned with the problem of minimizing a separable, convex, composite (smooth and nonsmooth) function subject to linear constraints. We study a randomized block-coordinate interpretation of the Chambolle-Pock primal-dual algorithm, based on inexact proximal gradient steps. A specificity of the considered algorithm is its robustness, as it converges even in the absence of strong duality or when the linear program is inconsistent. Using matrix preconditiong, we derive tight sublinear convergence rates with and without duality assumptions and for both the convex and the strongly convex settings. Our developments are extensions and particularizations of original algorithms proposed by Malitsky (2019) and Luke and Malitsky (2018). Numerical experiments are provided for an optimal transport problem of service pricing.
Paper Structure (15 sections, 5 theorems, 88 equations, 1 table, 2 algorithms)

This paper contains 15 sections, 5 theorems, 88 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Algorithm and main results
  3. Convergence analysis
  4. Interpretation as a proximal gradient algorithm
  5. ESO for block coordinate sampling
  6. Extrapolation
  7. Properties of $f$
  8. Main descent argument
  9. Duality and existence of a Lagrange multiplier
  10. Without a Lagrange multiplier
  11. Decreasing stepsizes for the strongly convex case
  12. Numerical experiments for an optimal transport problem
  13. Proofs of Theorems \ref{['theorem:convergence']} and \ref{['theorem:newscconvergence']}
  14. Convex case ($\Upsilon=0$)
  15. Strongly convex case

Key Result

Theorem 2.2

Let sequence $(x^{k})_{k}$ be issued by Algorithm algorithm:newgenericprimaldual with parameters $T^{k}\equiv T,\sigma^{k}\equiv\sigma$ satisfying earlydiagonalscaling, and consider the sequence $(s^{k})_{k}$ such that $s^{k}=\frac{1}{k}\sum_{l=1}^{k}x^{l}$.

Theorems & Definitions (10)

  • Theorem 2.2: Constant stepsize
  • Theorem 2.3: Strong convexity
  • Lemma 3.1: Proximal step
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • proof : Proof of Theorem \ref{['theorem:convergence']}
  • proof : Proof of Theorem \ref{['theorem:newscconvergence']}