Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Control, Optimal Transport and Neural Differential Equations in Supervised Learning

Minh-Nhat Phung, Minh-Binh Tran

TL;DR

This work addresses the challenge of approximating continuum unbalanced optimal transport (UOT) using Neural ODEs by introducing a constructive pipeline that starts from a Sinkhorn-type regularized formulation with Pearson divergence and builds up to a neural transport framework. It establishes a regularized primal–dual setup, provides explicit error bounds for the discretized Sinkhorn iterations, and derives a transport equation via a Monge–Ampère potential that links OT dynamics to a continuity/inhomogeneous transport system. The authors then connect this continuum OT to neural networks by constructing admissible vector fields from neural network parameters, proving convergence of the neural transport flows to the true UOT/OT dynamics in a controlled limiting regime, and showing how activations including ReLU and several others can realize the needed vector fields through neural approximate identities. The results yield a rigorous, constructive pathway to scalable transport-based learning with Neural ODEs, enabling explicit convergence guarantees and practical avenues for implementing OT-driven supervised learning and density estimation at scale.

Abstract

We study the fundamental computational problem of approximating optimal transport (OT) equations using neural differential equations (Neural ODEs). More specifically, we develop a novel framework for approximating unbalanced optimal transport (UOT) in the continuum using Neural ODEs. By generalizing a discrete UOT problem with Pearson divergence, we constructively design vector fields for Neural ODEs that converge to the true UOT dynamics, thereby advancing the mathematical foundations of computational transport and machine learning. To this end, we design a numerical scheme inspired by the Sinkhorn algorithm to solve the corresponding minimization problem and rigorously prove its convergence, providing explicit error estimates. From the obtained numerical solutions, we derive vector fields defining the transport dynamics and construct the corresponding transport equation. Finally, from the numerically obtained transport equation, we construct a neural differential equation whose flow converges to the true transport dynamics in an appropriate limiting regime.

Control, Optimal Transport and Neural Differential Equations in Supervised Learning

TL;DR

This work addresses the challenge of approximating continuum unbalanced optimal transport (UOT) using Neural ODEs by introducing a constructive pipeline that starts from a Sinkhorn-type regularized formulation with Pearson divergence and builds up to a neural transport framework. It establishes a regularized primal–dual setup, provides explicit error bounds for the discretized Sinkhorn iterations, and derives a transport equation via a Monge–Ampère potential that links OT dynamics to a continuity/inhomogeneous transport system. The authors then connect this continuum OT to neural networks by constructing admissible vector fields from neural network parameters, proving convergence of the neural transport flows to the true UOT/OT dynamics in a controlled limiting regime, and showing how activations including ReLU and several others can realize the needed vector fields through neural approximate identities. The results yield a rigorous, constructive pathway to scalable transport-based learning with Neural ODEs, enabling explicit convergence guarantees and practical avenues for implementing OT-driven supervised learning and density estimation at scale.

Abstract

We study the fundamental computational problem of approximating optimal transport (OT) equations using neural differential equations (Neural ODEs). More specifically, we develop a novel framework for approximating unbalanced optimal transport (UOT) in the continuum using Neural ODEs. By generalizing a discrete UOT problem with Pearson divergence, we constructively design vector fields for Neural ODEs that converge to the true UOT dynamics, thereby advancing the mathematical foundations of computational transport and machine learning. To this end, we design a numerical scheme inspired by the Sinkhorn algorithm to solve the corresponding minimization problem and rigorously prove its convergence, providing explicit error estimates. From the obtained numerical solutions, we derive vector fields defining the transport dynamics and construct the corresponding transport equation. Finally, from the numerically obtained transport equation, we construct a neural differential equation whose flow converges to the true transport dynamics in an appropriate limiting regime.

Paper Structure

This paper contains 27 sections, 9 theorems, 288 equations.

Table of Contents

  1. Introduction
  2. Settings and main results
  3. The problem of minimizing the total transport cost
  4. Dual problem
  5. Numerical Sinkhorn-type algorithm to solve the optimization problem
  6. Transport equation
  7. The Control and Optimal Transport Problem (B)
  8. Proofs of the main results
  9. Convex analysis
  10. The minimization algorithm
  11. Proof of Proposition \ref{['althm']}
  12. Proof of Proposition \ref{['eqthexactcapital']}
  13. Proof of Proposition \ref{['eqthmexact']}
  14. Proof of Theorem \ref{['neuralthm']}
  15. Some uniform bounds and approximation by Hermite polynomials
  16. ...and 12 more sections

Key Result

Lemma 2

For $f\mathcal{L},\, g\mathcal{L} \in \mathcal{M}^{\infty}_{\mathcal{L},c}(\mathbb{R}^d)$, the optimal transport cost is given by

Theorems & Definitions (24)

  • Definition 1
  • Remark 1
  • Lemma 2
  • Lemma 3
  • Remark 4
  • Proposition 5
  • Proposition 6
  • Remark 7
  • Proposition 8
  • Remark 9
  • ...and 14 more