Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Particle Algorithm for Mean-Field Variational Inference

Qiang Du, Kaizheng Wang, Edith Zhang, Chenyang Zhong

TL;DR

The paper introduces PArticle VI (PAVI), a particle-based mean-field variational inference method rooted in Wasserstein gradient flow and McKean–Vlasov dynamics. It delivers a non-asymptotic Wasserstein-2 bound between the particle approximation and the MFVI solution, valid for arbitrary numbers of particles and finite time, and decomposes error into initialization, particle discretization, time discretization, and stochastic gradient components. The authors extend the analysis to block MFVI, propose practical SGD-based implementations, and provide explicit guidance via corollaries for parameter choices; they also compare the particle-based approach to Langevin Monte Carlo in terms of convergence behavior. Overall, the work advances end-to-end theoretical guarantees for nonparametric MFVI and lays a foundation for efficient, scalable particle-based variational inference.

Abstract

Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing mean-field variational inference (MFVI) is coordinate ascent variational inference (CAVI), which relies crucially on parametric assumptions on complete conditionals. We introduce a novel particle-based algorithm for MFVI, named PArticle VI (PAVI), for nonparametric mean-field approximation. We obtain non-asymptotic error bounds for our algorithm. To our knowledge, this is the first end-to-end guarantee for particle-based MFVI.

A Particle Algorithm for Mean-Field Variational Inference

TL;DR

The paper introduces PArticle VI (PAVI), a particle-based mean-field variational inference method rooted in Wasserstein gradient flow and McKean–Vlasov dynamics. It delivers a non-asymptotic Wasserstein-2 bound between the particle approximation and the MFVI solution, valid for arbitrary numbers of particles and finite time, and decomposes error into initialization, particle discretization, time discretization, and stochastic gradient components. The authors extend the analysis to block MFVI, propose practical SGD-based implementations, and provide explicit guidance via corollaries for parameter choices; they also compare the particle-based approach to Langevin Monte Carlo in terms of convergence behavior. Overall, the work advances end-to-end theoretical guarantees for nonparametric MFVI and lays a foundation for efficient, scalable particle-based variational inference.

Abstract

Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing mean-field variational inference (MFVI) is coordinate ascent variational inference (CAVI), which relies crucially on parametric assumptions on complete conditionals. We introduce a novel particle-based algorithm for MFVI, named PArticle VI (PAVI), for nonparametric mean-field approximation. We obtain non-asymptotic error bounds for our algorithm. To our knowledge, this is the first end-to-end guarantee for particle-based MFVI.
Paper Structure (27 sections, 12 theorems, 89 equations, 2 algorithms)

This paper contains 27 sections, 12 theorems, 89 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related work
  4. Outline
  5. Notation
  6. Mean-field variational inference
  7. Problem setup
  8. Key properties of MFVI
  9. A particle algorithm for mean-field variational inference
  10. Fokker--Planck equations and diffusion processes
  11. PArticle VI
  12. Theoretical analysis
  13. Proofs
  14. Proof of \ref{['thm-main']}
  15. Proof of the claim regarding the particle approximation error
  16. ...and 12 more sections

Key Result

Lemma 2.1

$q_*$ is a fixed point of transform ${\mathcal{T}}:~ \mathcal{Q} \to \mathcal{Q}$, $q \mapsto {\mathcal{T}}_1 (q^{-1}) \otimes \cdots \otimes {\mathcal{T}}_m (q^{-m})$.

Theorems & Definitions (25)

  • Remark 1: Block MFVI
  • Definition 2.1: Entropy
  • Definition 2.2: Marginals
  • Definition 2.3: Approximate potential
  • Lemma 2.1: Fixed point characterization
  • Definition 3.1: 2-Wasserstein metric
  • Definition 3.2: Empirical measure of an array of particles
  • Remark 2
  • Remark 3: Block PAVI
  • Lemma 4.1: Theorems 1.1 of lacker2024mean and Theorem 2.3 of Lac23
  • ...and 15 more