Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parallelising Glauber dynamics

Holden Lee

TL;DR

This work studies sampling from distributions on discrete product spaces via Glauber dynamics and shows that parallelizing updates to a random subset of $k$ coordinates yields a near $k$-fold speedup in mixing, provided contraction and approximate tensorization of entropy hold. The analysis introduces a down-up operator framework that parallels proximal-sampler ideas and proves that $k$-Glauber contracts in $\mathcal{D}_{\\chi^2}$ by a factor ~$k$, and in KL-divergence under approximate tensorization. The authors then instantiate the framework for two spin systems: (i) Ising models with $\\|J\\|<1-c$, where each step of $\\Theta(n/\\|J\\|_F)$-Glauber can be implemented to yield running time $\\tilde{O}(\\|J\\|_F)=\\tilde{O}(\\sqrt{n})$; and (ii) high-temperature mixed $p$-spin models, where a $\\tilde{O}(\\sqrt{n})$-time parallel sampler is achieved with high probability. Across these cases, the results provide a general approach to designing fast parallel discrete samplers based on $k$-block updates, approximate rejection sampling, and entropy/tensorization tools, with practical implications for sampling Ising and mean-field spin-glass models.

Abstract

For distributions over discrete product spaces $\prod_{i=1}^n Ω_i'$, Glauber dynamics is a Markov chain that at each step, resamples a random coordinate conditioned on the other coordinates. We show that $k$-Glauber dynamics, which resamples a random subset of $k$ coordinates, mixes $k$ times faster in $χ^2$-divergence, and assuming approximate tensorization of entropy, mixes $k$ times faster in KL-divergence. We apply this to obtain parallel algorithms in two settings: (1) For the Ising model $μ_{J,h}(x)\propto \exp(\frac1 2\left\langle x,Jx \right\rangle + \langle h,x\rangle)$ with $\|J\|<1-c$ (the regime where fast mixing is known), we show that we can implement each step of $\widetilde Θ(n/\|J\|_F)$-Glauber dynamics efficiently with a parallel algorithm, resulting in a parallel algorithm with running time $\widetilde O(\|J\|_F) = \widetilde O(\sqrt n)$. (2) For the mixed $p$-spin model at high enough temperature, we show that with high probability we can implement each step of $\widetilde Θ(\sqrt n)$-Glauber dynamics efficiently and obtain running time $\widetilde O(\sqrt n)$.

Parallelising Glauber dynamics

TL;DR

This work studies sampling from distributions on discrete product spaces via Glauber dynamics and shows that parallelizing updates to a random subset of coordinates yields a near -fold speedup in mixing, provided contraction and approximate tensorization of entropy hold. The analysis introduces a down-up operator framework that parallels proximal-sampler ideas and proves that -Glauber contracts in by a factor ~, and in KL-divergence under approximate tensorization. The authors then instantiate the framework for two spin systems: (i) Ising models with , where each step of -Glauber can be implemented to yield running time ; and (ii) high-temperature mixed -spin models, where a -time parallel sampler is achieved with high probability. Across these cases, the results provide a general approach to designing fast parallel discrete samplers based on -block updates, approximate rejection sampling, and entropy/tensorization tools, with practical implications for sampling Ising and mean-field spin-glass models.

Abstract

For distributions over discrete product spaces , Glauber dynamics is a Markov chain that at each step, resamples a random coordinate conditioned on the other coordinates. We show that -Glauber dynamics, which resamples a random subset of coordinates, mixes times faster in -divergence, and assuming approximate tensorization of entropy, mixes times faster in KL-divergence. We apply this to obtain parallel algorithms in two settings: (1) For the Ising model with (the regime where fast mixing is known), we show that we can implement each step of -Glauber dynamics efficiently with a parallel algorithm, resulting in a parallel algorithm with running time . (2) For the mixed -spin model at high enough temperature, we show that with high probability we can implement each step of -Glauber dynamics efficiently and obtain running time .
Paper Structure (22 sections, 32 theorems, 90 equations, 4 algorithms)

This paper contains 22 sections, 32 theorems, 90 equations, 4 algorithms.

Table of Contents

  1. Introduction
  2. Main results
  3. Related work
  4. Continuous sampling
  5. Parallel algorithms for discrete sampling
  6. Diffusion models and the proximal sampler
  7. Preliminaries
  8. Markov kernels
  9. Functional inequalities
  10. Additional notation
  11. $k$-Glauber mixes $k$ times as fast
  12. Tensorization and projection
  13. Contraction improves going down
  14. Parallel sampling for Ising models
  15. Approximate tensorization of entropy for the Ising model
  16. ...and 7 more sections

Key Result

Theorem 1.1

Let $\mu$ be a distribution on $\Omega = \prod_{i=1}^{n} \Omega_i'$, and let $1\le k\le n$. Below, let $C\ge 1$.

Theorems & Definitions (65)

  • Theorem 1.1: $k$-Glauber mixes $k$ times faster
  • Theorem 1.2
  • Theorem 1.3
  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Definition 2.5
  • Definition 2.6
  • Definition 2.7
  • ...and 55 more