Control of probability flow in Markov chain Monte Carlo -- Nonreversibility and lifting

TL;DR

Several practical approaches to implementing nonreversible Markov chain Monte Carlo methods are reviewed, including the shift method in the cumulative distribution and the directed-worm algorithm.

Abstract

The Markov chain Monte Carlo (MCMC) method is widely used in various fields as a powerful numerical integration technique for systems with many degrees of freedom. In MCMC methods, probabilistic state transitions can be considered as a random walk in state space, and random walks allow for sampling from complex distributions. However, paradoxically, it is necessary to carefully suppress the randomness of the random walk to improve computational efficiency. By breaking detailed balance, we can create a probability flow in the state space and perform more efficient sampling along this flow. Motivated by this idea, practical and efficient nonreversible MCMC methods have been developed over the past ten years. In particular, the lifting technique, which introduces probability flows in an extended state space, has been applied to various systems and has proven more efficient than conventional reversible updates. We review and discuss several practical approaches to implementing nonreversible MCMC methods, including the shift method in the cumulative distribution and the directed-worm algorithm.

Figures (9)

• Figure 1: (a) Example of the weight tower for $n=6$ and (b) a periodic shift of the tower. $\pi_i$ is the weight of state $i$ and $F_i$ is the cumulative distribution. The parameter $s$ represents the shift amount of the weight tower. The stochastic flows and the transition probabilities are determined from the overlap between the original and shifted towers for each state. See the main text for the details. (c) When $s=\max_i \pi_i$, this approach reduces to the Suwa--Todo algorithm. suwa_markov_2010 This figure was taken from Ref. suwa_reducing_2024.
• Figure 2: Trajectories of configurations updated by the Gibbs sampler (left) and by the present nonreversible algorithm with $c=0.4$ and $w=0.1$ (right) in the bivariate Gaussian distribution with $\sigma_1=1$ and $\sigma_2=10$. The ellipsoidal line is the three-sigma line of the Gaussian distribution. The upper figures show the update procedures of each algorithm. This figure was taken from Ref. suwa_geometrically_2014.
• Figure 3: Autocorrelation times of $(x_1 + x_2 )^2$ in the bivariate Gaussian distribution by using the Gibbs sampler (triangles), the over-relaxation (circles) with $\alpha=-0.86$, the ordered over-relaxation (diamonds) with the number of candidates 10, and the shift method with $c=0.4$ and $w=0.05$ (squares). The horizontal axis $\sigma_2 / \sigma_1$ corresponds to the sampling difficulty. The statistical errors are in the same order as the point sizes.
• Figure 4: Multiple-proposal example for $n=4$. At first, a hub (pivot) is chosen from the current position $X$. Then, candidates $X'$, $X"$, and $X"'$ are generated from the hub. The dotted line shows the one-sigma line of the Gaussian distribution as a proposal example.
• Figure 5: Rejection rates (upper) and the correlation times of $(x_1+x_2)^2$ (lower) from the simple Metropolis algorithm and the rejection-minimized method for $n=3,4,5$ in the wine-bottle potential [\ref{['eqn:wb']}] with $h=16$. The rejection rate is reduced as the number of candidates is increased. Accompanying the rejection rate, the correlation time gets shorter.