Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scalable Simulation-free Entropic Unbalanced Optimal Transport

Jaemoo Choi, Jaewoong Choi

TL;DR

A scalable and simulation-free approach for solving the Entropic Unbalanced Optimal Transport (EUOT) problem, called Simulation-free EUOT (SF-EUOT), which demonstrates significantly improved scalability in generative modeling and image-to-image translation tasks compared to previous SB methods.

Abstract

The Optimal Transport (OT) problem investigates a transport map that connects two distributions while minimizing a given cost function. Finding such a transport map has diverse applications in machine learning, such as generative modeling and image-to-image translation. In this paper, we introduce a scalable and simulation-free approach for solving the Entropic Unbalanced Optimal Transport (EUOT) problem. We derive the dynamical form of this EUOT problem, which is a generalization of the Schrödinger bridges (SB) problem. Based on this, we derive dual formulation and optimality conditions of the EUOT problem from the stochastic optimal control interpretation. By leveraging these properties, we propose a simulation-free algorithm to solve EUOT, called Simulation-free EUOT (SF-EUOT). While existing SB models require expensive simulation costs during training and evaluation, our model achieves simulation-free training and one-step generation by utilizing the reciprocal property. Our model demonstrates significantly improved scalability in generative modeling and image-to-image translation tasks compared to previous SB methods.

Scalable Simulation-free Entropic Unbalanced Optimal Transport

TL;DR

A scalable and simulation-free approach for solving the Entropic Unbalanced Optimal Transport (EUOT) problem, called Simulation-free EUOT (SF-EUOT), which demonstrates significantly improved scalability in generative modeling and image-to-image translation tasks compared to previous SB methods.

Abstract

The Optimal Transport (OT) problem investigates a transport map that connects two distributions while minimizing a given cost function. Finding such a transport map has diverse applications in machine learning, such as generative modeling and image-to-image translation. In this paper, we introduce a scalable and simulation-free approach for solving the Entropic Unbalanced Optimal Transport (EUOT) problem. We derive the dynamical form of this EUOT problem, which is a generalization of the Schrödinger bridges (SB) problem. Based on this, we derive dual formulation and optimality conditions of the EUOT problem from the stochastic optimal control interpretation. By leveraging these properties, we propose a simulation-free algorithm to solve EUOT, called Simulation-free EUOT (SF-EUOT). While existing SB models require expensive simulation costs during training and evaluation, our model achieves simulation-free training and one-step generation by utilizing the reciprocal property. Our model demonstrates significantly improved scalability in generative modeling and image-to-image translation tasks compared to previous SB methods.
Paper Structure (54 sections, 7 theorems, 60 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 54 sections, 7 theorems, 60 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notations and Assumptions
  3. Background
  4. Optimal Transport Problems
  5. Kantorovich's Optimal Transport (OT)
  6. Entropic Optimal Transport (EOT)
  7. Schrödinger Bridge Problem and its Properties
  8. Schrödinger Bridge (SB) with Wiener prior
  9. Reciprocal Property of SB
  10. Equivalence between SB and EOT
  11. Dual Form of EOT
  12. Dynamcial and Dual Form of Entropic Unbalanced Optimal Transport
  13. Entropic Unbalanced Optimal Transport (EUOT)
  14. Dynamical Formulation of EUOT
  15. Dynamical Dual form of EUOT
  16. ...and 39 more sections

Key Result

Theorem 3.2

The EUOT problem is equivalent to the following dynamical transport problem: where $\partial_t \rho_t + \nabla \cdot (u_t \rho_t ) - \frac{\sigma^2}{2} \Delta \rho_t = 0$ and $\rho_0 = \mu$. Moreover, the optimal solution $\mathbb{P}^\star$ satisfies the reciprocal property, i.e.,

Figures (9)

  • Figure 1: Image Generation on CIFAR-10.
  • Figure 1: Unpaired Male → Female translation for 64 × 64 CelebA image.
  • Figure 2: Effect of Entropic Regularization $\sigma^2$ on Wild$\rightarrow$Cat (64x64).
  • Figure 3: Comparison on Benchmarks on High Dimensional Gaussian Experiments.$\Delta m$, $\Delta Var$, and $\Delta Cov$ stands for difference of mean, variance, and covariance, respectively. $m$, $Var$, and $Cov$ stands for ground true mean, variance and covariance, respectively.
  • Figure 4: Generated samples from our model trained on CIFAR-10 for $\sigma=0.1$.
  • ...and 4 more figures

Theorems & Definitions (14)

  • Remark 3.1: EUOT is a Generalization of EOT
  • Theorem 3.2
  • Remark 3.3: Dynamic form of EUOT is a Generalization of SB
  • Proposition 3.4: Dual formulation of EUOT
  • Lemma A.1
  • proof
  • Theorem A.2
  • proof
  • Proposition A.3: Dual I
  • proof
  • ...and 4 more