Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics

Fan Chen, Yiqing Lin, Zhenjie Ren, Songbo Wang

TL;DR

The kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional is studied and its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense is proved.

Abstract

We study the kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional. Using hypocoercivity, we first establish the exponential convergence of the mean field dynamics and then show the corresponding $N$-particle system converges exponentially in a rate uniform in $N$ modulo a small error. Finally we study the short-time regularization effects of the dynamics and prove its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense. Our results can be applied to the training of two-layer neural networks with momentum and we include the numerical experiments.

Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics

TL;DR

The kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional is studied and its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense is proved.

Abstract

We study the kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional. Using hypocoercivity, we first establish the exponential convergence of the mean field dynamics and then show the corresponding -particle system converges exponentially in a rate uniform in modulo a small error. Finally we study the short-time regularization effects of the dynamics and prove its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense. Our results can be applied to the training of two-layer neural networks with momentum and we include the numerical experiments.
Paper Structure (37 sections, 19 theorems, 220 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 37 sections, 19 theorems, 220 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Settings and main results
  3. Related works
  4. Coupling approaches.
  5. Functional approaches.
  6. Comparison to ulpoc.
  7. Main contributions
  8. Hypocoercivity for mean field systems.
  9. Regularization in short time.
  10. Propagation of chaos.
  11. Notations
  12. Organization of paper.
  13. Assumptions and main results
  14. Assumptions.
  15. Definition of $\hat{m}$ and functional inequalities.
  16. ...and 22 more sections

Key Result

Theorem 2.1

Assume $F$ satisfies eq:convexeq:lipeq:x-lsi. If $m_{0}$ has finite second moment, finite entropy and finite Fisher information, then there exist constants such that for every $t \geq 0$,

Figures (5)

  • Figure 1: Randomly chosen handwritten digits "$4$" and "$6$" from the MNIST dataset.
  • Figure 2: Individual (shadowed) and $10$-averaged (bold) losses without kinetic energy versus time.
  • Figure 3: Individual (shadowed) and $10$-averaged (bold) losses with kinetic energy versus time.
  • Figure 4: Average values of $\frac{1}{N}F_{\textnormal{NNet}} + \frac{1}{N}F_{\textnormal{Kinet}}$ over the last $500$ epochs. The mean (black squares) and standard derivations (error bars) are calculated from the $10$ independent runs. Dashed curve fits the data.
  • Figure 5: Target function $\frac{1}{N}F_{\textnormal{NNet}}$ for underdamped Langevin (blue) and overdamped Langevin (red) versus time.

Theorems & Definitions (45)

  • Remark 2.1
  • Theorem 2.1: Entropic convergence of MFL
  • Theorem 2.2: Entropic convergence of particle systems
  • Remark 2.2
  • Remark 2.3
  • Theorem 2.3: Wasserstein and entropic propagation of chaos
  • Remark 3.1
  • Lemma 4.1: Existence and uniqueness of invariant measures
  • Lemma 4.2: Mean field entropy sandwich
  • Lemma 4.3: Particle system's entropy inequality
  • ...and 35 more