Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sampling from Binary Quadratic Distributions via Stochastic Localization

Chenguang Wang, Kaiyuan Cui, Weichen Zhao, Tianshu Yu

TL;DR

This work addresses the challenge of sampling from binary quadratic distributions (BQDs) by applying stochastic localization (SL) to construct time-evolving posteriors that become increasingly easy to sample from as SL progresses. The authors prove that after a finite number of iterations the induced external field $h_t$ grows to infinity with high probability, which implies Poincaré inequalities and polynomial-time mixing for a broad class of discrete MCMC samplers, including Glauber dynamics and Metropolis-Hastings variants. The framework is demonstrated both theoretically and empirically: it covers warm-up and advanced DMCMC algorithms and yields consistent sampling improvements on MIS, MaxClique, and MaxCut across multiple datasets. This work thereby offers a principled, generalizable approach to accelerate discrete sampling tasks and broadens the applicability of SL in discrete domains, with potential impact on statistical physics, combinatorial optimization, and related machine learning applications.

Abstract

Sampling from binary quadratic distributions (BQDs) is a fundamental but challenging problem in discrete optimization and probabilistic inference. Previous work established theoretical guarantees for stochastic localization (SL) in continuous domains, where MCMC methods efficiently estimate the required posterior expectations during SL iterations. However, achieving similar convergence guarantees for discrete MCMC samplers in posterior estimation presents unique theoretical challenges. In this work, we present the first application of SL to general BQDs, proving that after a certain number of iterations, the external field of posterior distributions constructed by SL tends to infinity almost everywhere, hence satisfy Poincaré inequalities with probability near to 1, leading to polynomial-time mixing. This theoretical breakthrough enables efficient sampling from general BQDs, even those that may not originally possess fast mixing properties. Furthermore, our analysis, covering enormous discrete MCMC samplers based on Glauber dynamics and Metropolis-Hastings algorithms, demonstrates the broad applicability of our theoretical framework. Experiments on instances with quadratic unconstrained binary objectives, including maximum independent set, maximum cut, and maximum clique problems, demonstrate consistent improvements in sampling efficiency across different discrete MCMC samplers.

Sampling from Binary Quadratic Distributions via Stochastic Localization

TL;DR

This work addresses the challenge of sampling from binary quadratic distributions (BQDs) by applying stochastic localization (SL) to construct time-evolving posteriors that become increasingly easy to sample from as SL progresses. The authors prove that after a finite number of iterations the induced external field grows to infinity with high probability, which implies Poincaré inequalities and polynomial-time mixing for a broad class of discrete MCMC samplers, including Glauber dynamics and Metropolis-Hastings variants. The framework is demonstrated both theoretically and empirically: it covers warm-up and advanced DMCMC algorithms and yields consistent sampling improvements on MIS, MaxClique, and MaxCut across multiple datasets. This work thereby offers a principled, generalizable approach to accelerate discrete sampling tasks and broadens the applicability of SL in discrete domains, with potential impact on statistical physics, combinatorial optimization, and related machine learning applications.

Abstract

Sampling from binary quadratic distributions (BQDs) is a fundamental but challenging problem in discrete optimization and probabilistic inference. Previous work established theoretical guarantees for stochastic localization (SL) in continuous domains, where MCMC methods efficiently estimate the required posterior expectations during SL iterations. However, achieving similar convergence guarantees for discrete MCMC samplers in posterior estimation presents unique theoretical challenges. In this work, we present the first application of SL to general BQDs, proving that after a certain number of iterations, the external field of posterior distributions constructed by SL tends to infinity almost everywhere, hence satisfy Poincaré inequalities with probability near to 1, leading to polynomial-time mixing. This theoretical breakthrough enables efficient sampling from general BQDs, even those that may not originally possess fast mixing properties. Furthermore, our analysis, covering enormous discrete MCMC samplers based on Glauber dynamics and Metropolis-Hastings algorithms, demonstrates the broad applicability of our theoretical framework. Experiments on instances with quadratic unconstrained binary objectives, including maximum independent set, maximum cut, and maximum clique problems, demonstrate consistent improvements in sampling efficiency across different discrete MCMC samplers.

Paper Structure

This paper contains 63 sections, 12 theorems, 150 equations, 2 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. Discrete MCMC Samplers
  5. SL for Sampling
  6. Binary Quadratic Sampling via Stochastic Localization
  7. Physical Intuition
  8. Theoretical Analysis
  9. Setup and Key Assumptions
  10. Warm-up Cases: Poincaré Inequality for Glauber Dynamics and Metropolis Chains
  11. Glauber Dynamics
  12. Metropolis Chains.
  13. Poincaré Inequality for Advanced DMCMC Algorithms
  14. Single-Site Metropolis-Hastings Algorithms
  15. DULA
  16. ...and 48 more sections

Key Result

Theorem 3.1

Consider the observation process in the case $\frac{\alpha(t)}{\sigma\sqrt{t}}\rightarrow+\infty$ as $t\rightarrow+\infty$, for arbitrary $\zeta>0$ and $\varepsilon>0$, there is a $T(\zeta,\varepsilon)$ large enough such that for $t\geq T$, the observation process satisfies

Figures (2)

  • Figure 1: Ablation study comparing two design choices: (1) MCMC steps allocation strategies (Exponential Decay vs. Identical) and (2) SDE time discretization methods (Uniform vs. Log-SNR). Hatched bars indicate the best performing configuration for each algorithm-dataset combination.
  • Figure 2: Sampling trajectories comparison between MCMC samplers (dash lines) and their SL-based counterparts (solid lines) across different datasets. The shaded areas in the trajectories, representing the variance across multiple instances

Theorems & Definitions (22)

  • Theorem 3.1
  • Remark 4.2
  • Theorem 4.3
  • Corollary 4.4
  • Theorem 4.5
  • Theorem 4.6
  • Remark 4.7
  • Theorem 4.8
  • Theorem 4.9
  • Remark 4.10
  • ...and 12 more