Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Sublinear Convergence Rates for Iterative Bregman Projections

Gabriel Peyré

TL;DR

This work develops a general dual-analytic framework for cyclic KL projections (iterative Bregman projections) onto two affine blocks under entropic regularization. It proves a robust $O(1/(\gamma k))$ dual convergence rate that crucially depends on a block-quotient dual seminorm, enabling stable complexity bounds for approximating unregularized linear programs as the regularization parameter $\gamma$ shrinks. The authors extend the theory to general Bregman divergences with a generalized Pinsker condition and provide bias bounds linking entropic and unregularized optima. They showcase practical impact by deriving a Flow-Sinkhorn algorithm for Wasserstein-1 on graphs with $O(p/\varepsilon^4)$ operation count, and they discuss non-expansiveness properties that support uniform bounds on dual iterates, making the approach broadly applicable beyond classical OT.

Abstract

Entropic regularization provides a simple way to approximate linear programs whose constraints split into two (or more) tractable blocks. The resulting objectives are amenable to cyclic Kullback-Leibler (KL) Bregman projections, with the classical Sinkhorn algorithm for optimal transport (balanced, unbalanced, gradient flows, barycenters, \dots) as the canonical example. Assuming uniformly bounded primal mass and dual radius, we prove that the dual objective of these KL projections decreases at an $O(1/k)$ rate with a constant that scales only linearly in $1/γ$, where $γ$ is the entropic regularization parameter. This extends the guarantees known for entropic optimal transport to any such linearly constrained problem. Following the terminology introduced in [Chizat et al 2025], we call such rates "robust", because this mild dependence on $γ$ underpins favorable complexity bounds for approximating the unregularized problem via alternating KL projections. The crucial aspect of the analysis is that the dual radius should be measured according to a block-quotient dual seminorm, which depends on the structure of the split of the constraint into blocks. As an application, we derive the flow-Sinkhorn algorithm for the Wasserstein-1 distance on graphs. It achieves $ε$-additive accuracy on the transshipment cost in $O(p/ε^{4})$ arithmetic operations, where $p$ is the number of edges.

Robust Sublinear Convergence Rates for Iterative Bregman Projections

TL;DR

This work develops a general dual-analytic framework for cyclic KL projections (iterative Bregman projections) onto two affine blocks under entropic regularization. It proves a robust dual convergence rate that crucially depends on a block-quotient dual seminorm, enabling stable complexity bounds for approximating unregularized linear programs as the regularization parameter shrinks. The authors extend the theory to general Bregman divergences with a generalized Pinsker condition and provide bias bounds linking entropic and unregularized optima. They showcase practical impact by deriving a Flow-Sinkhorn algorithm for Wasserstein-1 on graphs with operation count, and they discuss non-expansiveness properties that support uniform bounds on dual iterates, making the approach broadly applicable beyond classical OT.

Abstract

Entropic regularization provides a simple way to approximate linear programs whose constraints split into two (or more) tractable blocks. The resulting objectives are amenable to cyclic Kullback-Leibler (KL) Bregman projections, with the classical Sinkhorn algorithm for optimal transport (balanced, unbalanced, gradient flows, barycenters, \dots) as the canonical example. Assuming uniformly bounded primal mass and dual radius, we prove that the dual objective of these KL projections decreases at an rate with a constant that scales only linearly in , where is the entropic regularization parameter. This extends the guarantees known for entropic optimal transport to any such linearly constrained problem. Following the terminology introduced in [Chizat et al 2025], we call such rates "robust", because this mild dependence on underpins favorable complexity bounds for approximating the unregularized problem via alternating KL projections. The crucial aspect of the analysis is that the dual radius should be measured according to a block-quotient dual seminorm, which depends on the structure of the split of the constraint into blocks. As an application, we derive the flow-Sinkhorn algorithm for the Wasserstein-1 distance on graphs. It achieves -additive accuracy on the transshipment cost in arithmetic operations, where is the number of edges.
Paper Structure (45 sections, 24 theorems, 138 equations)

This paper contains 45 sections, 24 theorems, 138 equations.

Table of Contents

  1. Introduction
  2. Entropic regularization of structured linear programs
  3. Previous works
  4. Iterative Bregman projection.
  5. Entropic regularisation and Sinkhorn-type methods.
  6. Wasserstein-1 on graphs.
  7. Contributions
  8. Iterative KL Projections
  9. Entropic regularisation and cyclic KL projections
  10. Dual point of view
  11. Dual problem.
  12. Block–coordinate ascent in the dual.
  13. First–order conditions.
  14. Dual convergence
  15. Main result
  16. ...and 30 more sections

Key Result

Proposition 2.1

Let $z\in\mathbb{R}^d_{++}$ be fixed and set $Z \coloneqq \sum_{i=1}^d z_i$. Define the dual objective, for $u\in\mathbb{R}^m$, Then Moreover, any maximiser $u^\star$ and the unique primal minimiser $x^\star$ are linked by and $u^\star$ is characterised by the stationarity condition

Theorems & Definitions (51)

  • Proposition 2.1: Dual of \ref{['eq:entropic-penalized']}
  • Definition 3.1: block-quotient seminorms
  • Theorem 3.1: Sub–linear dual rate
  • proof
  • Lemma 3.1: Per–step ascent for the dual blocks
  • proof
  • Lemma 3.2: Dual gap versus global residual
  • proof
  • Lemma 3.3: KL bias
  • proof
  • ...and 41 more