Efficient Sampling for Ising and Potts Models using Auxiliary Gaussian Variables

Abstract

Ising and Potts models are an important class of discrete probability distributions which originated from statistical physics and since then have found applications in several disciplines. Simulation from these models is a well known challenging problem. In this paper, we study a class of Markov chain Monte Carlo algorithms, in which we introduce an auxiliary Gaussian variable such that, conditional on this variable, the discrete states are independent. This approach is broadly applicable to Ising and Potts models, including ones in which the coupling matrix admits negative entries, as in spin glass and Hopfield models. We focus on a block Gibbs sampler version of this algorithm, which alternates between sampling the auxiliary Gaussian and the discrete states, and derive mixing time bounds for a wide class of Ising/Potts models at both high and low temperatures, yielding results analogous to those derived for the Heat Bath and Swendsen-Wang algorithms. We present novel choices of auxiliary Gaussian variables which scale well with the number of states in the Potts model, and which can take advantage of the low rank structure of the coupling matrix, if any. Finally, we numerically evaluate the performance of the auxiliary Gaussian Gibbs sampler with several competing algorithms, across a range of examples.

Figures (5)

• Figure 1: $\widehat{R}$ over time. Results on a $24 \times 24$ Ising grid at or near the critical temperature $\beta_c = 1 / \sqrt{2} \approx 0.707$. At various checkpoints, we evaluate the run time and $\widehat{R}$ for each sampler. $\widehat{R} \le 1.1$ indicates the sampler is nearing convergence; in practice the more conservative threshold $\widehat{R} \le 1.01$ is recommended. Overall, the AG sampler achieves the fastest convergence, as measured by $\widehat{R}$.
• Figure 2: Effective sample size per second on a $24 \times 24$ grid. (Above) Ising model, i.e. $q = 2$, and the critical temperature is $\beta_c = 1 / \sqrt{2}$. (Below) Potts model with $q = 4$ and the critical temperature is $\beta_c = 0.55$.
• Figure 3: Effective sample size per second on a $24 \times 24$ fully-connected graph. (Above) Ising model, i.e. $q = 2$. The critical temperature occurs at $\beta_c = 1$ and incurs a decrease in efficiency for all algorithms. Using a low-rank AG improves performance. (Below) For $q = 4$, we observe a critical slow down for all algorithms near the critical temperature $\beta_c \sim 1.65$. We also obtain competitive performance for a large $\beta$ (cold temperatures).
• Figure 4: Effective sample size per second on a $24 \times 24$ Hopfield model. For a low-rank coupling matrix, the low-rank AG offers a large improvement in performance. For larger ranks, the regular AG sampler performs slightly better. For higher ranks, the performance of all algorithms decreases.
• Figure 5: Effective sample size per second for a Spin Glass system with $n = 128$. At cold temperatures (high $\beta$), the tempered scheme improves over the non-tempered AG sampler. However the advantage of the AG sampler over the Heat Bath vanishes for $\beta \ge 2$.

Theorems & Definitions (23)

• Remark 1
• Remark 2
• Proposition 3
• Proposition 4
• Theorem 5
• Remark 6
• Remark 7
• Theorem 8
• Theorem 9
• Remark 10