Accelerated Markov Chain Monte Carlo Algorithms on Discrete States

TL;DR

This paper addresses the challenge of rapid sampling on finite discrete state spaces by embedding Nesterov-style acceleration into Metropolis-Hastings through a damped Hamiltonian flow on the probability simplex endowed with a discrete Wasserstein-2 metric. It develops a family of accelerated discrete MCMC dynamics (aMCMC) built from various divergences and mobilities, notably Chi-squared, KL, log-Fisher, and con-Fisher, and proves positivity and (under suitable conditions) Z-free convergence guarantees. The authors introduce practical numerical schemes including a staggered ODE integrator and a jump-process discretization, along with warm-start and restart strategies to maintain stability, and demonstrate faster convergence and higher accuracy (often O(1/M)) compared to MH on graphs such as lattices, hypercubes, and Gaussian mixtures. The work advances discrete-score-inspired sampling by marrying variational transport on graphs with momentum-based optimization, enabling efficient, normalization-free sampling for complex discrete distributions with potential impact on Bayesian inference and statistical physics simulations.

Abstract

We propose a class of discrete state sampling algorithms based on Nesterov's accelerated gradient method, which extends the classical Metropolis-Hastings (MH) algorithm. The evolution of the discrete states probability distribution governed by MH can be interpreted as a gradient descent direction of the Kullback--Leibler (KL) divergence, via a mobility function and a score function. Specifically, this gradient is defined on a probability simplex equipped with a discrete Wasserstein-2 metric with a mobility function. This motivates us to study a momentum-based acceleration framework using damped Hamiltonian flows on the simplex set, whose stationary distribution matches the discrete target distribution. Furthermore, we design an interacting particle system to approximate the proposed accelerated sampling dynamics. The extension of the algorithm with a general choice of potentials and mobilities is also discussed. In particular, we choose the accelerated gradient flow of the relative Fisher information, demonstrating the advantages of the algorithm in estimating discrete score functions without requiring the normalizing constant and keeping positive probabilities. Numerical examples, including sampling on a Gaussian mixture supported on lattices or a distribution on a hypercube, demonstrate the effectiveness of the proposed discrete-state sampling algorithm.

Figures (6)

• Figure 1: Graph configurations employed in the numerical experiments.
• Figure 2: Sampling on $C_3$ graph \ref{['fig:C3']} via Chi-squared method. x-axes are in iterations and all results are over time span $[0,65]$ when both ODE solvers achieve machine precision. (a) Validation of the Chi-squared method using step size $\Delta t=0.1$, showing density convergence and the decay of $\log_{10}(\mathcal{H}(p(t),\psi(t)))$. (b) and (c) show the decay of $\log_{10}\lVert p(t)-\pi\rVert_2$ for step sizes $\Delta t=0.1$ and $\Delta t=0.01$ respectively. Notably, chi-jump achieves an error of order $\mathcal{O}(1/M)$.
• Figure 3: Sampling on a two-loop graph of node 8 (see \ref{['fig:2loop']}) via log-Fisher method. x-axes are in iterations with the step size $\Delta t=0.1$. y-axes is on error $\log_{10}\lVert p(t)-\pi\rVert_2$. The right figure zooms in the first 120 iterations of the left figure. The jump process via log-Fisher method exhibits faster convergence and higher accuracy.
• Figure 4: Sampling on a hypercube graph of 64 nodes (see \ref{['fig:hypercube']}) via log-Fisher method. x-axes are in iterations with step size $\Delta t= 0.01$. The left figure shows the approximation error $\log_{10}\lVert p(t)-\pi\rVert_2$ w.r.t the target distribution. The right figure shows the approximation error $\log_{10}\lvert\sum_{i=1}^n p_i(t)\log \frac{p_i(t)}{Z\pi } - (-\log Z)\rvert$ w.r.t the normalizing constant $Z$.
• Figure 5: Sampling on a $25\times 25$ lattice graph (see \ref{['fig:lattice']}) via log-Fisher method for the target distribution as a mixture of two Gaussian distributions. x-axes are in iterations. The left figure shows the approximation error $\log_{10}\lVert p(t)-\pi\rVert_2$ w.r.t the target distribution. The right figure shows the approximation error $\log_{10}\lvert\sum_{i=1}^n p_i(t)\log \frac{p_i(t)}{Z\pi } - (-\log Z)\rvert$ w.r.t the normalizing constant $Z$. The jump process via log-Fisher achieves to a higher accuracy when that via MH is approaching to $\mathcal{O}(\frac{1}{\sqrt{M}})$.

Theorems & Definitions (13)

• Lemma 1
• Theorem 2
• Example 1
• Example 2
• Theorem 3
• Lemma 4
• Theorem 5
• Remark 6
• Lemma 7
• Lemma 8