Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contraction and entropy production in continuous-time Sinkhorn dynamics

Anand Srinivasan, Jean-Jacques Slotine

TL;DR

The paper addresses contraction and entropy production in continuous-time Sinkhorn dynamics for entropic OT. It develops two time-dependent metrics derived from the mirror Hessian to establish contraction under a coercivity condition, and proves an exact entropy-production rate identity, linking the Sinkhorn flow to Onsager gradient structures and a nonlocal Dirichlet form. Key contributions include a Poincaré inequality with a uniform spectral gap for $\varepsilon>0$, and showing that exponential entropy decay is equivalent to a log-Sobolev inequality, with consequences for convergence guarantees. The results yield practical guidance for latent-space design in generative modeling and provide stopping heuristics for discrete Sinkhorn iterations, enhancing both theory and application of entropic OT methods.

Abstract

Recently, the vanishing-step-size limit of the Sinkhorn algorithm at finite regularization parameter $\varepsilon$ was shown to be a mirror descent in the space of probability measures. We give $L^2$ contraction criteria in two time-dependent metrics induced by the mirror Hessian, which reduce to the coercivity of certain conditional expectation operators. We then give an exact identity for the entropy production rate of the Sinkhorn flow, which was previously known only to be nonpositive. Examining this rate shows that the standard semigroup analysis of diffusion processes extends systematically to the Sinkhorn flow. We show that the flow induces a reversible Markov dynamics on the target marginal as an Onsager gradient flow. We define the Dirichlet form associated to its (nonlocal) infinitesimal generator, prove a Poincaré inequality for it, and show that the spectral gap is strictly positive along the Sinkhorn flow whenever $\varepsilon > 0$. Lastly, we show that the entropy decay is exponential if and only if a logarithmic Sobolev inequality (LSI) holds. We give for illustration two immediate practical use-cases for the Sinkhorn LSI: as a design principle for the latent space in which generative models are trained, and as a stopping heuristic for discrete-time algorithms.

Contraction and entropy production in continuous-time Sinkhorn dynamics

TL;DR

The paper addresses contraction and entropy production in continuous-time Sinkhorn dynamics for entropic OT. It develops two time-dependent metrics derived from the mirror Hessian to establish contraction under a coercivity condition, and proves an exact entropy-production rate identity, linking the Sinkhorn flow to Onsager gradient structures and a nonlocal Dirichlet form. Key contributions include a Poincaré inequality with a uniform spectral gap for , and showing that exponential entropy decay is equivalent to a log-Sobolev inequality, with consequences for convergence guarantees. The results yield practical guidance for latent-space design in generative modeling and provide stopping heuristics for discrete Sinkhorn iterations, enhancing both theory and application of entropic OT methods.

Abstract

Recently, the vanishing-step-size limit of the Sinkhorn algorithm at finite regularization parameter was shown to be a mirror descent in the space of probability measures. We give contraction criteria in two time-dependent metrics induced by the mirror Hessian, which reduce to the coercivity of certain conditional expectation operators. We then give an exact identity for the entropy production rate of the Sinkhorn flow, which was previously known only to be nonpositive. Examining this rate shows that the standard semigroup analysis of diffusion processes extends systematically to the Sinkhorn flow. We show that the flow induces a reversible Markov dynamics on the target marginal as an Onsager gradient flow. We define the Dirichlet form associated to its (nonlocal) infinitesimal generator, prove a Poincaré inequality for it, and show that the spectral gap is strictly positive along the Sinkhorn flow whenever . Lastly, we show that the entropy decay is exponential if and only if a logarithmic Sobolev inequality (LSI) holds. We give for illustration two immediate practical use-cases for the Sinkhorn LSI: as a design principle for the latent space in which generative models are trained, and as a stopping heuristic for discrete-time algorithms.
Paper Structure (8 sections, 6 theorems, 68 equations)

This paper contains 8 sections, 6 theorems, 68 equations.

Table of Contents

  1. Introduction
  2. Background
  3. Entropic optimal transport
  4. Sinkhorn algorithm for ${\rm OT}_\varepsilon$ as an $L^2$ mirror descent
  5. Notation
  6. Definitions
  7. Assumptions
  8. Results

Key Result

Theorem 1

The Sinkhorn flow eq:sinkhorn_dual, eq:sinkhorn_primal is contracting (or expanding) with rate $\lambda \in \mathbb{R}$ in the time-dependent metric for all states $\pi_t$ and tangent directions $\xi \in \ker Q_{\pi_t}$ at which the conditional expectation operator $P_{\pi_t}$ defined in eq:P_pi satisfies the coercivity property

Theorems & Definitions (23)

  • Definition 2.1: Conditional expectation operator
  • Definition 2.2: Numerical range
  • Definition 2.3: Coercivity
  • Theorem 1: Contraction of Sinkhorn flow in $\langle \cdot, \cdot\rangle_{1/\pi_t^2}$
  • proof
  • Remark 1
  • Theorem 2: Contraction of Sinkhorn flow in Fisher-Rao
  • proof
  • Remark 2
  • Remark 3
  • ...and 13 more