Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sampling from Mean-Field Gibbs Measures via Diffusion Processes

Ahmed El Alaoui, Andrea Montanari, Mark Sellke

TL;DR

The paper tackles efficient sampling from mean-field Ising mixed p-spin Gibbs measures at high temperature by turning sampling into a diffusion-based process guided by stochastic localization. The core idea is to approximate the mean of tilted Gibbs measures with AMP (state evolution guarantees) and to refine it via a TAP-based natural gradient descent, embedded in a discretized diffusion that asymptotically samples from the target μ_G. The authors prove that, for β below a computable β_bar(ξ), the algorithm outputs samples whose law is W2-close to the true Gibbs measure with near-linear gradient complexity, and they provide hardness results ruling out stable samplers in regimes with replica-symmetry breaking or shattering. The work extends diffusion-based sampling to general mixed p-spin models (beyond SK), connects to denoising diffusion processes, and establishes a sharp separation between algorithmic feasibility and the physics-driven instability phenomena via disorder chaos. Together these contributions yield a principled, scalable framework for sampling in challenging high-dimensional nonconvex spin-glass landscapes with explicit quantitative guarantees.

Abstract

We consider Ising mixed $p$-spin glasses at high-temperature and without external field, and study the problem of sampling from the Gibbs distribution $μ$ in polynomial time. We develop a new sampling algorithm with complexity of the same order as evaluating the gradient of the Hamiltonian and, in particular, at most linear in the input size. We prove that, at sufficiently high-temperature, it produces samples from a distribution $μ^{alg}$ which is close in normalized Wasserstein distance to $μ$. Namely, there exists a coupling of $μ$ and $μ^{alg}$ such that if $({\boldsymbol x},{\boldsymbol x}^{alg})\in\{-1,+1\}^n\times \{-1,+1\}^n$ is a pair drawn from this coupling, then $n^{-1}{\mathbb E}\{\|{\boldsymbol x}-{\boldsymbol x}^{alg}\|_2^2\}=o_n(1)$. For the case of the Sherrington-Kirkpatrick model, our algorithm succeeds in the full replica-symmetric phase. We complement this result with a negative one for sampling algorithms satisfying a certain `stability' property, which is verified by many standard techniques. No stable algorithm can approximately sample at temperatures below the onset of shattering, even under the normalized Wasserstein metric. Further, no algorithm can sample at temperatures below the onset of replica symmetry breaking. Our sampling method implements a discretized version of a diffusion process that has become recently popular in machine learning under the name of `denoising diffusion.' We derive the same process from the general construction of stochastic localization. Implementing the diffusion process requires to efficiently approximate the mean of the tilted measure. To this end, we use an approximate message passing algorithm that, as we prove, achieves sufficiently accurate mean estimation.

Sampling from Mean-Field Gibbs Measures via Diffusion Processes

TL;DR

The paper tackles efficient sampling from mean-field Ising mixed p-spin Gibbs measures at high temperature by turning sampling into a diffusion-based process guided by stochastic localization. The core idea is to approximate the mean of tilted Gibbs measures with AMP (state evolution guarantees) and to refine it via a TAP-based natural gradient descent, embedded in a discretized diffusion that asymptotically samples from the target μ_G. The authors prove that, for β below a computable β_bar(ξ), the algorithm outputs samples whose law is W2-close to the true Gibbs measure with near-linear gradient complexity, and they provide hardness results ruling out stable samplers in regimes with replica-symmetry breaking or shattering. The work extends diffusion-based sampling to general mixed p-spin models (beyond SK), connects to denoising diffusion processes, and establishes a sharp separation between algorithmic feasibility and the physics-driven instability phenomena via disorder chaos. Together these contributions yield a principled, scalable framework for sampling in challenging high-dimensional nonconvex spin-glass landscapes with explicit quantitative guarantees.

Abstract

We consider Ising mixed -spin glasses at high-temperature and without external field, and study the problem of sampling from the Gibbs distribution in polynomial time. We develop a new sampling algorithm with complexity of the same order as evaluating the gradient of the Hamiltonian and, in particular, at most linear in the input size. We prove that, at sufficiently high-temperature, it produces samples from a distribution which is close in normalized Wasserstein distance to . Namely, there exists a coupling of and such that if is a pair drawn from this coupling, then . For the case of the Sherrington-Kirkpatrick model, our algorithm succeeds in the full replica-symmetric phase. We complement this result with a negative one for sampling algorithms satisfying a certain `stability' property, which is verified by many standard techniques. No stable algorithm can approximately sample at temperatures below the onset of shattering, even under the normalized Wasserstein metric. Further, no algorithm can sample at temperatures below the onset of replica symmetry breaking. Our sampling method implements a discretized version of a diffusion process that has become recently popular in machine learning under the name of `denoising diffusion.' We derive the same process from the general construction of stochastic localization. Implementing the diffusion process requires to efficiently approximate the mean of the tilted measure. To this end, we use an approximate message passing algorithm that, as we prove, achieves sufficiently accurate mean estimation.
Paper Structure (32 sections, 46 theorems, 293 equations, 3 algorithms)

This paper contains 32 sections, 46 theorems, 293 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background: Sampling from spin glass Gibbs measures
  3. Background: Sampling via diffusion processes
  4. Summary of results
  5. Conference version.
  6. Notations
  7. Main results: I. Sampling algorithm for $\beta$ small enough
  8. Approximating the mean of the Gibbs measure
  9. Sampling via stochastic localization
  10. Theoretical guarantee
  11. Main results: II. Hardness of stable sampling from RSB or shattering
  12. Properties of stochastic localization
  13. Analysis of the sampling algorithm and proof of Theorem \ref{['thm:main']}
  14. The planted model and contiguity
  15. Approximate Message Passing
  16. ...and 17 more sections

Key Result

Theorem 2.1

Define the functions $h(x):=-((1+x)/2)\log((1+x)/2) - ((1+x)/2)\log((1+x)/2)$, $\psi(\gamma) = \operatorname{\mathbb{E}}[\tanh(\gamma+\sqrt{\gamma} \, G)]$ (with expectation with respect to $G\sim{\sf N}(0,1)$), and $\phi=\psi^{-1}$ its inverse. Let $\bar{\beta}(\xi):=\min(\beta_1(\xi),\beta_2(\xi), depend uniquely on the mixture polynomial $\xi$ (here $\widehat{\xi}^{(\ell)}(1) :=\sum_{p=2}^Pc_p

Theorems & Definitions (103)

  • Remark 1.1
  • Theorem 2.1
  • Remark 2.1
  • Remark 2.2: Condition $\beta<\beta_1$
  • Remark 2.3: SK model
  • Remark 2.4: Pure $p$-spin
  • Definition 3.1
  • Theorem 3.2: Stability of the sampling Algorithm \ref{['alg:Sampling']}
  • Corollary 3.3
  • proof
  • ...and 93 more