Parallelize Single-Site Dynamics up to Dobrushin Criterion
Hongyang Liu, Yitong Yin
TL;DR
The paper tackles the challenge of parallelizing single-site Markov chain sampling, aiming to faithfully simulate Glauber-type dynamics in parallel. It introduces a locally-iterative, BP-like algorithm that uses universal coupling (correlated sampling) and analyzed Dobrushin-type conditions to achieve depth bounds; notably, depth is O(N/n + log n) for N steps (on tilde-O(m) processors) under a relaxed norm-condition, and exponentially faster when the domain is Boolean and the norm is strictly less than 1. The authors derive RNC samplers for hardcore and Ising models in uniqueness regimes and an RNC SAT sampler in a local lemma regime, with non-adaptive simulated annealing enabling RNC counting. They also provide distributed CONGEST implementations and discuss reducing total work by time-partitioning. Overall, the work demonstrates that near-linear-mixing single-site dynamics can be parallelized into RNC algorithms under mild correlation-control conditions, with broad implications for scalable sampling and counting in graphical models and SAT problems.
Abstract
Single-site dynamics are canonical Markov chain based algorithms for sampling from high-dimensional distributions, such as the Gibbs distributions of graphical models. We introduce a simple and generic parallel algorithm that faithfully simulates single-site dynamics. Under a much relaxed, asymptotic variant of the $\ell_p$-Dobrushin's condition -- where the Dobrushin's influence matrix has a bounded $\ell_p$-induced operator norm for an arbitrary $p\in[1, \infty]$ -- our algorithm simulates $N$ steps of single-site updates within a parallel depth of $O\left({N}/{n}+\log n\right)$ on $\tilde{O}(m)$ processors, where $n$ is the number of sites and $m$ is the size of the graphical model. For Boolean-valued random variables, if the $\ell_p$-Dobrushin's condition holds -- specifically, if the $\ell_p$-induced operator norm of the Dobrushin's influence matrix is less than~$1$ -- the parallel depth can be further reduced to $O(\log N+\log n)$, achieving an exponential speedup. These results suggest that single-site dynamics with near-linear mixing times can be parallelized into $\mathsf{RNC}$ sampling algorithms, independent of the maximum degree of the underlying graphical model, as long as the Dobrushin influence matrix maintains a bounded operator norm. We show the effectiveness of this approach with $\mathsf{RNC}$ samplers for the hardcore and Ising models within their uniqueness regimes, as well as an $\mathsf{RNC}$ SAT sampler for satisfying solutions of CNF formulas in a local lemma regime. Furthermore, by employing non-adaptive simulated annealing, these $\mathsf{RNC}$ samplers can be transformed into $\mathsf{RNC}$ algorithms for approximate counting.