Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sampling from the Continuous Random Energy Model in Total Variation Distance

Holden Lee, Qiang Wu

TL;DR

This work studies efficient sampling from the CREM Gibbs measure on a binary tree in the high-temperature regime, establishing two polynomial-time algorithms (Markov chain and sequential) that achieve total-variation proximity to the target distribution. The results hinge on precise partition-function concentration, additive and multiplicative approximations of the free energy, and a detailed analysis of tree-based conductance and tilt-contiguity of tilted CREMs, together with a sharp bound showing the spectral gap is exponentially small. A key contribution is showing TV guarantees up to $\beta_{\min}=\min\{\beta_c,\beta_G\}$, with concave $A$ allowing up to $\beta_c$ and non-concave $A$ matching the algorithmic threshold $\beta_G$, thereby completing the picture for CREM sampling in several regimes. The findings illuminate inherent limitations of Markov-chain methods due to bottlenecks, while providing a viable algorithmic avenue via additive/multiplicative control and change-of-measure techniques, with potential implications for broader spin-glass models such as the SK model. The work also introduces a contiguity concept for tilted CREMs that may be useful beyond CREM and offers quantitative tools for finite-depth sequential sampling.

Abstract

The continuous random energy model (CREM) is a toy model of spin glasses on $\{0,1\}^N$ that, in the limit, exhibits an infinitely hierarchical correlation structure. We give two polynomial-time algorithms to approximately sample from the Gibbs distribution of the CREM in the high-temperature regime $β<β_{\min}:=\min\{β_c,β_G\}$, based on a Markov chain and a sequential sampler. The running time depends algebraically on the desired TV distance and failure probability and exponentially in $(1/g)^{O(1)}$, where $g$ is the gap to a certain inverse temperature threshold $β_{\min}$; this contrasts with previous results which only attain $o(N)$ accuracy in KL divergence. If the covariance function $A$ of the CREM is concave, the algorithms work up to the critical threshold $β_c$, which is the static phase transition point; while for $A$ non-concave, if $β_G<β_c$, the algorithms work up to the known algorithmic threshold $β_G$ proposed in Addario-Berry and Maillard (2020) for non-trivial sampling guarantees. Our result depends on quantitative bounds for the fluctuation of the partition function and a new contiguity result of the ``tilted" CREM obtained from sampling, which is of independent interest. We also show that the spectral gap is exponentially small with high probability, suggesting that the algebraic dependence is unavoidable with a Markov chain approach.

Sampling from the Continuous Random Energy Model in Total Variation Distance

TL;DR

This work studies efficient sampling from the CREM Gibbs measure on a binary tree in the high-temperature regime, establishing two polynomial-time algorithms (Markov chain and sequential) that achieve total-variation proximity to the target distribution. The results hinge on precise partition-function concentration, additive and multiplicative approximations of the free energy, and a detailed analysis of tree-based conductance and tilt-contiguity of tilted CREMs, together with a sharp bound showing the spectral gap is exponentially small. A key contribution is showing TV guarantees up to , with concave allowing up to and non-concave matching the algorithmic threshold , thereby completing the picture for CREM sampling in several regimes. The findings illuminate inherent limitations of Markov-chain methods due to bottlenecks, while providing a viable algorithmic avenue via additive/multiplicative control and change-of-measure techniques, with potential implications for broader spin-glass models such as the SK model. The work also introduces a contiguity concept for tilted CREMs that may be useful beyond CREM and offers quantitative tools for finite-depth sequential sampling.

Abstract

The continuous random energy model (CREM) is a toy model of spin glasses on that, in the limit, exhibits an infinitely hierarchical correlation structure. We give two polynomial-time algorithms to approximately sample from the Gibbs distribution of the CREM in the high-temperature regime , based on a Markov chain and a sequential sampler. The running time depends algebraically on the desired TV distance and failure probability and exponentially in , where is the gap to a certain inverse temperature threshold ; this contrasts with previous results which only attain accuracy in KL divergence. If the covariance function of the CREM is concave, the algorithms work up to the critical threshold , which is the static phase transition point; while for non-concave, if , the algorithms work up to the known algorithmic threshold proposed in Addario-Berry and Maillard (2020) for non-trivial sampling guarantees. Our result depends on quantitative bounds for the fluctuation of the partition function and a new contiguity result of the ``tilted" CREM obtained from sampling, which is of independent interest. We also show that the spectral gap is exponentially small with high probability, suggesting that the algebraic dependence is unavoidable with a Markov chain approach.
Paper Structure (23 sections, 30 theorems, 174 equations, 1 figure, 3 algorithms)

This paper contains 23 sections, 30 theorems, 174 equations, 1 figure, 3 algorithms.

Table of Contents

  1. Introduction
  2. The continuous random energy model
  3. Main results
  4. Background and related works
  5. Notation
  6. Structure of the paper
  7. Free energy and partition function approximation
  8. Fluctuation of free energy and additive approximation
  9. Free energy computation and multiplicative approximation
  10. Markov chain sampler
  11. Conductance on trees
  12. Case 1.
  13. Case 2.
  14. Tail bounds for $\widehat{Z}_{\beta, N}$
  15. Bounding $s$-conductance for the CREM
  16. ...and 8 more sections

Key Result

Theorem 1.4

Given Assumption a:hi-temp(i:1) ($\beta<\beta_{\min}$), with probability at least $1-\delta$ over the CREM, to generate a sample from a distribution $\widehat{\mu}$ such that $\operatorname{TV}(\widehat{\mu},\mu_{\beta, N})<\varepsilon$,

Figures (1)

  • Figure 1: Example of the CREM tree with $N=5$. All vertices are denoted as strings of 0s and 1s. The length of string corresponds to the depth of the vertex. In the above graph, for vertices $v=10000$ and $w=10011$, by definition $v \wedge w=100$ denotes the longest common ancestor vertex.

Theorems & Definitions (68)

  • Definition 1.1
  • Remark 1.2
  • Theorem 1.4
  • Remark 1.5
  • Theorem 1.6: Upper bound for spectral gap
  • Theorem 1.7
  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • ...and 58 more