Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Numerical Analysis on Neural Network Projected Schemes for Approximating One Dimensional Wasserstein Gradient Flows

Xinzhe Zuo, Jiaxi Zhao, Shu Liu, Stanley Osher, Wuchen Li

TL;DR

A numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows and derives a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure.

Abstract

We provide a numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows. We approximate the Lagrangian mapping functions of gradient flows by the class of two-layer neural network functions with ReLU (rectified linear unit) activation functions. The numerical scheme is based on a projected gradient method, namely the Wasserstein natural gradient, where the projection is constructed from the $L^2$ mapping spaces onto the neural network parameterized mapping space. We establish theoretical guarantees for the performance of the neural projected dynamics. We derive a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure. General truncation error analysis is also established on the basis of the projective nature of the dynamics. Numerical examples, including gradient drift Fokker-Planck equations, porous medium equations, and Keller-Segel models, verify the accuracy and effectiveness of the proposed neural projected algorithm.

Numerical Analysis on Neural Network Projected Schemes for Approximating One Dimensional Wasserstein Gradient Flows

TL;DR

A numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows and derives a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure.

Abstract

We provide a numerical analysis and computation of neural network projected schemes for approximating one dimensional Wasserstein gradient flows. We approximate the Lagrangian mapping functions of gradient flows by the class of two-layer neural network functions with ReLU (rectified linear unit) activation functions. The numerical scheme is based on a projected gradient method, namely the Wasserstein natural gradient, where the projection is constructed from the mapping spaces onto the neural network parameterized mapping space. We establish theoretical guarantees for the performance of the neural projected dynamics. We derive a closed-form update for the scheme with well-posedness and explicit consistency guarantee for a particular choice of network structure. General truncation error analysis is also established on the basis of the projective nature of the dynamics. Numerical examples, including gradient drift Fokker-Planck equations, porous medium equations, and Keller-Segel models, verify the accuracy and effectiveness of the proposed neural projected algorithm.
Paper Structure (11 sections, 3 theorems, 54 equations, 1 table, 1 algorithm)

This paper contains 11 sections, 3 theorems, 54 equations, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Review of Wasserstein gradient flows and Lagrangian coordinates
  3. Wasserstein gradient flows
  4. Lagrangian coordinates & Particle dynamics
  5. Neural projected Wassersetin gradient flows and their algorithms
  6. Neural network activation functions
  7. Neural mapping models and energies
  8. Neural mapping metric space
  9. Neural mapping dynamics
  10. Neural projected Wasserstein flows
  11. Algorithm

Key Result

Proposition 1

The gradient operator of $F$ in $(\Theta, G_{\mathrm{W}})$, $\mathrm{grad}_{\mathrm{W}}F(\theta)=(\mathrm{grad}_{\mathrm{W}} F(\theta)_k)_{k=1}^D$, is given by

Theorems & Definitions (15)

  • Example 1: Linear
  • Example 2: ReLU
  • Example 3: Sigmoid
  • Definition 1: Neural mapping models
  • Definition 2: Neural mapping energies
  • Definition 3: Neural mapping distance
  • Definition 4: Neural mapping metric
  • Proposition 1: Neural mapping gradient operators
  • proof
  • Proposition 2: Neural mapping gradient flows
  • ...and 5 more