Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sampling from the Mean-Field Stationary Distribution

Yunbum Kook, Matthew S. Zhang, Sinho Chewi, Murat A. Erdogdu, Mufan Bill Li

TL;DR

We address the problem of sampling from the stationary distribution of mean-field SDEs by decoupling the two core steps: (i) approximating the mean-field dynamics with a finite-particle system via uniform-in-time propagation of chaos and (ii) sampling from the finite-particle stationary distribution using standard log-concave samplers. The authors establish three propagation-of-chaos results in $W_2$, $\mathsf{KL}$, and $\mathsf{FI}$ metrics, and prove a uniform-in-$N$ log-Sobolev inequality for the mean-field Langevin stationary distribution. This modular framework yields improved algorithmic guarantees, including tighter bounds for training certain two-layer neural networks in the mean-field regime, and provides practical paths to combine advanced sampling algorithms with recent chaos bounds. The work also develops verifiable isoperimetric properties for the stationary measures, enabling reliable application of log-concave samplers to mean-field targets, while highlighting open challenges in achieving uniformly good LSI constants in $N$-scaling.

Abstract

We study the complexity of sampling from the stationary distribution of a mean-field SDE, or equivalently, the complexity of minimizing a functional over the space of probability measures which includes an interaction term. Our main insight is to decouple the two key aspects of this problem: (1) approximation of the mean-field SDE via a finite-particle system, via uniform-in-time propagation of chaos, and (2) sampling from the finite-particle stationary distribution, via standard log-concave samplers. Our approach is conceptually simpler and its flexibility allows for incorporating the state-of-the-art for both algorithms and theory. This leads to improved guarantees in numerous settings, including better guarantees for optimizing certain two-layer neural networks in the mean-field regime. A key technical contribution is to establish a new uniform-in-$N$ log-Sobolev inequality for the stationary distribution of the mean-field Langevin dynamics.

Sampling from the Mean-Field Stationary Distribution

TL;DR

We address the problem of sampling from the stationary distribution of mean-field SDEs by decoupling the two core steps: (i) approximating the mean-field dynamics with a finite-particle system via uniform-in-time propagation of chaos and (ii) sampling from the finite-particle stationary distribution using standard log-concave samplers. The authors establish three propagation-of-chaos results in , , and metrics, and prove a uniform-in- log-Sobolev inequality for the mean-field Langevin stationary distribution. This modular framework yields improved algorithmic guarantees, including tighter bounds for training certain two-layer neural networks in the mean-field regime, and provides practical paths to combine advanced sampling algorithms with recent chaos bounds. The work also develops verifiable isoperimetric properties for the stationary measures, enabling reliable application of log-concave samplers to mean-field targets, while highlighting open challenges in achieving uniformly good LSI constants in -scaling.

Abstract

We study the complexity of sampling from the stationary distribution of a mean-field SDE, or equivalently, the complexity of minimizing a functional over the space of probability measures which includes an interaction term. Our main insight is to decouple the two key aspects of this problem: (1) approximation of the mean-field SDE via a finite-particle system, via uniform-in-time propagation of chaos, and (2) sampling from the finite-particle stationary distribution, via standard log-concave samplers. Our approach is conceptually simpler and its flexibility allows for incorporating the state-of-the-art for both algorithms and theory. This leads to improved guarantees in numerous settings, including better guarantees for optimizing certain two-layer neural networks in the mean-field regime. A key technical contribution is to establish a new uniform-in- log-Sobolev inequality for the stationary distribution of the mean-field Langevin dynamics.
Paper Structure (32 sections, 23 theorems, 154 equations, 1 table, 1 algorithm)

This paper contains 32 sections, 23 theorems, 154 equations, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions and Organization
  3. Related Work
  4. Background and Notation
  5. SDE Systems and Their Stationary Distributions
  6. The Pairwise McKean--Vlasov Setting
  7. The General McKean--Vlasov Setting
  8. Technical Ingredients
  9. Bias Control via Uniform-in-Time Propagation of Chaos
  10. Pairwise McKean--Vlasov Setting
  11. General McKean--Vlasov Setting
  12. Isoperimetric Properties of the Stationary Distributions
  13. Pairwise McKean--Vlasov Setting
  14. General McKean--Vlasov Setting
  15. Sampling from the Mean-Field Target
  16. ...and 17 more sections

Key Result

Theorem 3

Under Assumptions as:smoothness, as:pi_lsi and as:weak_interaction, for any $N \ge 100$ and $k \in [N]$, it holds that $\mathsf{KL}(\mu^{1:k} \mathbin{\|} \pi^{\otimes k}) = \widetilde{\mathcal{O}}(dk^2/N^2)$. Thus, $\mathsf{KL}(\mu^{1:k} \mathbin{\|} \pi^{\otimes k}) < \varepsilon^2$ if

Theorems & Definitions (29)

  • Definition 1
  • Definition 2: Log-Sobolev Inequality
  • Theorem 3: Sharp Propagation of Chaos
  • Theorem 4: Weak Propagation of Chaos
  • Theorem 5: Propagation of Chaos for General Functionals
  • Lemma 6: Informal
  • Lemma 7
  • Lemma 8
  • Lemma 9: Informal
  • Example 10: Gaussian Case
  • ...and 19 more