Reshaping Global Loop Structure to Accelerate Local Optimization by Smoothing Rugged Landscapes

TL;DR

This work introduces a structured M-layer lifting of factor graphs, where a tunable mixing kernel Q shapes global loop structure without altering local interactions. The authors show, via cavity theory and 1-RSB analysis, that inter-layer fluctuations contract and that the lifted dynamics perform a stochastic descent on the base graph’s Bethe free energy, effectively smoothing rugged landscapes. Empirically, the approach improves MAP inference and reduces compute-to-solution costs across Ising benchmarks, while enhancing replica-exchange performance for the maximum independent set problem. The framework is general, compatible with standard local-update algorithms, and opens avenues for instance-specific mixing designs to accelerate convergence in optimization and probabilistic decoding tasks. Overall, structured M-layer mixing provides a practical, topology-driven mechanism to tame metastability and accelerate convergence in complex graphical models.

Abstract

Probabilistic graphical models with frustration exhibit rugged energy landscapes that trap iterative optimization dynamics. These landscapes are shaped not only by local interactions, but crucially also by the global loop structure of the graph. The famous Bethe approximation treats the graph as a tree, effectively ignoring global structure, thereby limiting its effectiveness for optimization. Loop expansions capture such global structure in principle, but are often impractical due to combinatorial explosion. The $M$-layer construction provides an alternative: make $M$ copies of the graph and reconnect edges between them uniformly at random. This provides a controlled sequence of approximations from the original graph at $M=1$, to the Bethe approximation as $M \rightarrow \infty$. Here we generalize this construction by replacing uniform random rewiring with a structured mixing kernel $Q$ that sets the probability that any two layers are interconnected. As a result, the global loop structure can be shaped without modifying local interactions. We show that, after this copy-and-reconnect transformation, there exists a regime in which layer-to-layer fluctuations decay, increasing the probability of reaching the global minimum of the energy function of the original graph. This yields a highly general and practical tool for optimization. Using this approach, the computational cost required to reach these optimal solutions is reduced across sparse and dense Ising benchmarks, including spin glasses and planted instances. When combined with replica-exchange Monte Carlo, the same construction increases the polynomial-time algorithmic threshold for the maximum independent set problem. A cavity analysis shows that structured inter-layer coupling significantly smooths rugged energy landscapes by collapsing configurational complexity and suppressing many suboptimal metastable states.

Figures (10)

• Figure 1: Schematic of the structured $M$-layer lift. a) $M$-layer for a 1D chain factor graph. The original graph is replicated $M=3$ times, with the structured graphs interconnected by directed connections permuted across layers. The quench disorder remains identical in every layer. b) Structured permutation, where the probability of permutations between blocks $a$ and $b$ is set by the mixing matrix $Q_{ab}$ in the case of an Ising model. c) $M$-layer lift obtained from a directed 1D ring $Q$ such as Gaussian circulant, illustrating that the $M$-layer lift can generate cycles in the layer-to-layer graph.
• Figure 2: Zero-temperature quenches of the $M$-layer–lifted graph. a) Mixing kernel $Q_{ab}$. In this example, $Q$ is Gaussian circulant matrix with mean shift $\mu$ and width $\sigma$. b) and d) Scaling of mean convergence iterations and residual energy $\langle e - e_0 \rangle$, respectively, at optimal $\sigma^*$ with layer count $M$ for different shift values $\mu$. c) Residual energy vs. mixing kernel width $\sigma$. $L=1$. $N=50$. $T=2 \times 10^5$ maximum sweeps.
• Figure 3: Decrease of residual energy with the number of layers $M$. a) Minimum over the mixing-kernel width $\sigma$ of the mean residual energy per spin $\langle e - e_0 \rangle$ of Ising random regular graphs of degree $k=3$, obtained by averaging $K$ runs (and $10$ instances) of the greedy zero temperature Glauber update; vertical bars denote the stored 95% confidence intervals, and dotted curves show the fitted power laws $a M^p$. b) and c) Fitted exponent $p$ versus $N$ and prefactor $a$ versus $N$ on a logarithmic scale. The KS $p$-values are 0.157, 0.104, and 0.295 for $N=60$, $90$, and $120$, respectively. $K=50$. $L=2$.
• Figure 4: Computational advantage of the $M$-layer lift. a) Operation-to-target (OTT) at optimal $\sigma^*$ versus layer count $M$. b) Scaling of OTT$^*$ at optimal $M$ vs. base problem size $N$. c) Layer count $M$ that achieves the optimal OTT for each problem and solver. The OTT combines per-run computational cost (operations) $d N M T$ with the number of runs required for a $99\%$ success probability. Error bars show $95\%$ confidence intervals over disorder realizations. Instances are random regular graphs of degree $d=3$. Glauber ($T=0$) = zero-temperature Glauber updates. SA = simulated annealing. RRG = random regular graph of degree $3$. SK = Sherrington-Kirkpatrick with weights $\pm 1$. Tile planted are dimension 2 with patterns $(p_1,p_2,p_3) = (0.2,0.5,0.1)$. Block size is $L=1$. The apparent saturation of the RRG/Glauber ($T=0$) curve at large $M \geq 14$ reflects the finite range of tested layer counts rather than an intrinsic limitation of the method.
• Figure 5: Algorithmic thresholds for MIS under replica–exchange methods. The main panel shows the mean number of Monte Carlo sweeps (MCS) required to reach independent-set density $\rho$ for $\mu$SA, $\mu$RSA, $\mu$PT, and their $M$-layer extensions (M-layer $\mu$SA with $M=5$, M-layer $\mu$PT with $M=3$). Points denote empirical measurements, and solid curves give power-law fits $\tau(\rho)=C(\rho_{\rm alg}-\rho)^{-\nu}$. Vertical dashed lines indicate the algorithmic thresholds $\rho_{\rm alg}$, with shaded regions showing 95% confidence intervals. The inset summarizes the fitted $\rho_{\rm alg}$ values. The structured $M$-layer lift elevates the threshold of $\mu$SA to that of $\mu$PT and further improves $\mu$PT itself, yielding the highest threshold among all compared algorithms. $\mu$PT uses 30 replicas, M-layer $\mu$PT uses 22 and 12 replicas for $M=2$ and $M=3$, respectively.