Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low Stein Discrepancy via Message-Passing Monte Carlo

Nathan Kirk, T. Konstantin Rusch, Jakob Zech, Daniela Rus

TL;DR

This work addresses sampling from general multivariate distributions with known densities by learning low-discrepancy samples that minimize the kernel Stein discrepancy (KSD). It extends the Message-Passing Monte Carlo (MPMC) framework to Stein-MPMC, using a graph neural network to transform an initial point set into a low-KSD configuration relative to $F$, with the objective given by $D_{\mathcal{H}_0,F}({X_i}) = \sqrt{ \frac{1}{N^2} \sum_{i,j} k_0(X_i,X_j) }$. Empirical results on a Gaussian Mixture and a Beta Product distribution show Stein-MPMC attains smaller KSD than both SVGD and Stein Points, indicating superior sample quality for a fixed $N$. The approach suggests promising scalability to higher dimensions and can benefit from adaptive kernel tuning to further enhance performance.

Abstract

Message-Passing Monte Carlo (MPMC) was recently introduced as a novel low-discrepancy sampling approach leveraging tools from geometric deep learning. While originally designed for generating uniform point sets, we extend this framework to sample from general multivariate probability distributions with known probability density function. Our proposed method, Stein-Message-Passing Monte Carlo (Stein-MPMC), minimizes a kernelized Stein discrepancy, ensuring improved sample quality. Finally, we show that Stein-MPMC outperforms competing methods, such as Stein Variational Gradient Descent and (greedy) Stein Points, by achieving a lower Stein discrepancy.

Low Stein Discrepancy via Message-Passing Monte Carlo

TL;DR

This work addresses sampling from general multivariate distributions with known densities by learning low-discrepancy samples that minimize the kernel Stein discrepancy (KSD). It extends the Message-Passing Monte Carlo (MPMC) framework to Stein-MPMC, using a graph neural network to transform an initial point set into a low-KSD configuration relative to , with the objective given by . Empirical results on a Gaussian Mixture and a Beta Product distribution show Stein-MPMC attains smaller KSD than both SVGD and Stein Points, indicating superior sample quality for a fixed . The approach suggests promising scalability to higher dimensions and can benefit from adaptive kernel tuning to further enhance performance.

Abstract

Message-Passing Monte Carlo (MPMC) was recently introduced as a novel low-discrepancy sampling approach leveraging tools from geometric deep learning. While originally designed for generating uniform point sets, we extend this framework to sample from general multivariate probability distributions with known probability density function. Our proposed method, Stein-Message-Passing Monte Carlo (Stein-MPMC), minimizes a kernelized Stein discrepancy, ensuring improved sample quality. Finally, we show that Stein-MPMC outperforms competing methods, such as Stein Variational Gradient Descent and (greedy) Stein Points, by achieving a lower Stein discrepancy.

Paper Structure

This paper contains 14 sections, 14 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Stein Discrepancy
  4. Stein-Message-Passing Monte Carlo (Stein-MPMC)
  5. Stein-MPMC Model
  6. Results
  7. Experimental Detail
  8. Gaussian Mixture Distribution
  9. Beta Product Distribution
  10. Discussion
  11. Training Details
  12. Stein-MPMC
  13. Stein Points
  14. Stein Variational Gradient Descent

Figures (2)

  • Figure 1: Schematic of the MPMC model. The points are encoded to a high dimensional representation then passed through multiple layers of a message-passing GNN where the underlying computational graph is constructed based on nearest neighbors. Finally, the node-wise output representations of the final layer of the GNN are decoded.
  • Figure 2: KSD results for our two target distributions. Stein-MPMC yields smaller KSD values for every $N = 20, 60, 100, \ldots, 500$ across both distributions.