Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transport alpha divergences

Wuchen Li

Abstract

We derive a class of divergences measuring the difference between probability density functions on the one-dimensional sample space. This divergence is a one-parameter variation of the Itakura--Saito divergence between quantile density functions. We prove that the proposed divergence is a one-parameter variation of the transport Kullback-Leibler divergence and the Hessian distance of negative Boltzmann entropy with respect to the Wasserstein-$2$ metric. From Taylor expansions, we also formulate the $3$-symmetric tensor in Wasserstein-$2$ space, which is given by an iterative Gamma three operator. The alpha--geodesic on Wasserstein space is also derived. From these properties, we name the proposed divergences transport alpha divergences. We provide several examples of transport alpha divergences on one dimensional distributions, such as generative models and Cauchy distributions.

Transport alpha divergences

Abstract

We derive a class of divergences measuring the difference between probability density functions on the one-dimensional sample space. This divergence is a one-parameter variation of the Itakura--Saito divergence between quantile density functions. We prove that the proposed divergence is a one-parameter variation of the transport Kullback-Leibler divergence and the Hessian distance of negative Boltzmann entropy with respect to the Wasserstein- metric. From Taylor expansions, we also formulate the -symmetric tensor in Wasserstein- space, which is given by an iterative Gamma three operator. The alpha--geodesic on Wasserstein space is also derived. From these properties, we name the proposed divergences transport alpha divergences. We provide several examples of transport alpha divergences on one dimensional distributions, such as generative models and Cauchy distributions.

Paper Structure

This paper contains 16 sections, 13 theorems, 128 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Divergence functions and Information geometry methods
  3. Transport alpha divergences
  4. Review of Wasserstein-$2$ distances
  5. Transport alpha divergences
  6. Properties
  7. Transport alpha geodesic
  8. Hessian structures of entropy in Wasserstein-$2$ space
  9. Review
  10. Transport 3--symmetric tensor
  11. Examples
  12. Discussion
  13. Derivatives in Wasserstein-$2$ space
  14. Transport Hessian distance
  15. Transport KL divergence
  16. ...and 1 more sections

Key Result

Proposition 1

The following equation holds:

Figures (1)

  • Figure 1: Three curves for $\partial_xT_\alpha(t,x)$ (left), and $\sigma_\alpha(t)$ (right) with $\sigma_p=1$, $\sigma_q=5$. Red: $\alpha=1$. Black: $\alpha=0$. Blue $\alpha=-1$.

Theorems & Definitions (41)

  • Definition 1: Transport alpha divergence
  • Proposition 1
  • proof
  • Remark 1
  • Proposition 2: Positivity and Duality
  • proof
  • Proposition 3: Taylor expansions in Wasserstein-$2$ space
  • proof
  • Theorem 1: Alpha--Itakura--Saito divergences in Wasserstein-$2$ space
  • proof
  • ...and 31 more