Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transport information Bregman divergences

Wuchen Li

TL;DR

This work extends the concept of Bregman divergences to the $L^2$-Wasserstein probability-density space by introducing transport Bregman divergences $\mathrm{D}_{\mathrm{T},\mathcal{F}}$ and the transport KL divergence $\mathrm{D}_{\mathrm{TKL}}$ as a KL-like object in Wasserstein geometry. It establishes a general formulation via the optimal transport map $T$ and $\nabla_x\Phi_p$, provides multiple equivalent representations (mapping-based and joint-density-based), and derives tractable closed-form expressions in one dimension and for Gaussian families. The paper proves key properties such as nonnegativity under displacement convexity, a transport Hessian metric, linearity, and asymmetry, and introduces a symmetrized transport Jensen–Shannon divergence. These results illuminate how information-geometric notions interact with optimal transport and point to practical applications in AI inference and optimization within Wasserstein spaces.

Abstract

We study Bregman divergences in probability density space embedded with the $L^2$-Wasserstein metric. Several properties and dualities of transport Bregman divergences are provided. In particular, we derive the transport Kullback-Leibler (KL) divergence by a Bregman divergence of negative Boltzmann-Shannon entropy in $L^2$-Wasserstein space. We also derive analytical formulas and generalizations of transport KL divergence for one-dimensional probability densities and Gaussian families.

Transport information Bregman divergences

TL;DR

This work extends the concept of Bregman divergences to the -Wasserstein probability-density space by introducing transport Bregman divergences and the transport KL divergence as a KL-like object in Wasserstein geometry. It establishes a general formulation via the optimal transport map and , provides multiple equivalent representations (mapping-based and joint-density-based), and derives tractable closed-form expressions in one dimension and for Gaussian families. The paper proves key properties such as nonnegativity under displacement convexity, a transport Hessian metric, linearity, and asymmetry, and introduces a symmetrized transport Jensen–Shannon divergence. These results illuminate how information-geometric notions interact with optimal transport and point to practical applications in AI inference and optimization within Wasserstein spaces.

Abstract

We study Bregman divergences in probability density space embedded with the -Wasserstein metric. Several properties and dualities of transport Bregman divergences are provided. In particular, we derive the transport Kullback-Leibler (KL) divergence by a Bregman divergence of negative Boltzmann-Shannon entropy in -Wasserstein space. We also derive analytical formulas and generalizations of transport KL divergence for one-dimensional probability densities and Gaussian families.

Paper Structure

This paper contains 17 sections, 14 theorems, 110 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Bregman divergences in Euclidean space
  3. Bregman divergences in $L^2$--Wasserstein space
  4. Review of $L^2$--Wasserstein space
  5. Transport Bregman divergence
  6. Formulations
  7. Examples
  8. Properties
  9. Transport KL divergence
  10. Review of KL divergence
  11. Transport KL divergence
  12. Examples in Gaussian distributions
  13. Transport Jenson--Shannon divergence
  14. Discussion
  15. Proofs in section \ref{['sec3']}
  16. ...and 2 more sections

Key Result

Proposition 1

The following facts hold.

Figures (2)

  • Figure 1: A comparison between KL divergence and transport KL divergence for one dimensional Gaussian distributions. Left represents the KL divergence. Right represents the transport KL divergence.
  • Figure 2: Illustration of transport Jensen--Shannon divergences for one dimensional Gaussian distributions.

Theorems & Definitions (46)

  • Definition 1: Bregman divergence
  • Definition 2: Transport coordinates
  • Proposition 1: Facts in $L^2$--Wasserstein space AGSVillani2009_optimal
  • Definition 3: Transport Bregman divergence
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Remark 1: Connections with Euclidean Bregman divergences
  • Remark 2
  • ...and 36 more