Finding the right path: statistical mechanics of connected solutions in constraint satisfaction problems

TL;DR

This work introduces a local-entropy biased statistical mechanics framework to characterize connected solution manifolds in constraint satisfaction problems, using the symmetric binary perceptron (SBP) as a case study. By constructing a multi-layer local-entropy bias and applying a no-memory replica Ansatz, it reveals a delocalized, star-shaped cluster of connected minima whose edge-to-core structure remains dynamically accessible above a critical threshold $\kappa^{\text{no-mem}}_{\text{loc. stab.}}$ but destabilizes below it. The analysis connects with Franz-Parisi potentials in simpler settings and demonstrates, via a tailored Monte Carlo algorithm, that solutions persist up to the predicted threshold, aligning theory with simulations. Overall, the approach provides a principled way to study dynamical accessibility in rugged landscapes and informs the design of algorithms targeting connected regions, with potential applications to broader CSPs and evolution-like systems.

Abstract

We define and study a statistical mechanics ensemble that characterizes connected solutions in constraint satisfaction problems (CSPs). Built around a well-known local entropy bias, it allows us to better identify hardness transitions in problems where the energy landscape is dominated by isolated solutions. We apply this new device to the symmetric binary perceptron model (SBP), and study how its manifold of connected solutions behaves. We choose this particular problem because, while its typical solutions are isolated, it can be solved using local algorithms for a certain range of constraint density $α$ and threshold $κ$. With this new ensemble, we unveil the presence of a cluster composed of delocalized connected solutions. In particular, we demonstrate its stability until a critical threshold $κ^{\rm no-mem}_{\rm loc.\, stab.}$ (dependent on $α$). This transition appears as paths of solutions shatter, a phenomenon that more conventional statistical mechanics approaches fail to grasp. Finally, we compared our predictions to simulations. For this, we used a modified Monte-Carlo algorithm, designed specifically to target these delocalized solutions. We obtained, as predicted, that the algorithm finds solutions until $κ\approxκ^{\rm no-mem}_{\rm loc.\, stab.}$.

Figures (15)

• Figure 1: Drawing representing the local arrangement of solutions $\bf x$ around a robust reference minima $\bf x_0$. At any extensive distance from ${\bf x}_0$ -${\bf x}\cdot{\bf x}_0/N=\mathcal{O}(1)$-, typical solutions are isolated as infinite energy barriers surround them krauth89storage. This known as an overlap-gap property (OGP) gamarnik2021survey. Due to these barriers, these solutions ${\bf x}$ cannot be accessed with local algorithms. Extremely close to ${\bf x}_0$ -for ${\bf x}\!\cdot\!{\bf x}_0/\!N\!=\!o(1)$-, paths of connected solutions are dominated in number by Markov chains Barbier2025. Such paths can be explored by a classical Monte-Carlo algorithm, but their length is not extensive. This work proposes to find the atypical paths (pictured in orange) that would allow us to decorrelate from ${\bf x}_0$.
• Figure 2: Drawing representing the connected structure introduced to evaluate $\phi$ -see Eq. (\ref{['eq: quenched free energ connected cluster']})-. On the top, we have the generic structure where all configurations can couple with each other. The only constraint is that each configuration ${\bf x}^{{\bf P}_k}$ overlaps with its direct ancestor: ${\bf x}^{{\bf P}^*_k}\cdot{\bf x}^{{\bf P}_k}/N=m$. At the bottom, we detailed how almost all correlating fields are dropped once the no-memory $Ansatz$ is taken.
• Figure 3: Drawing representing a star-shaped cluster of the connected minima. To obtain this structure within our formalism, we applied two successive simplifications: first, the no-memory $Ansatz$, and second, the chain arrangement ($y_k=1$ for all $k\in[\![1,k_f]\!]$). This cluster can be divided into two main components: an edge and a core. The typical minima are contained in the edge, they are the least robust ones in the cluster. To pass from one to the other, our statistical measure goes through the core of the cluster -where the minima are the most robust-.
• Figure 4: Representation of the perturbation in the no-memory cluster(s). While along a no-memory path the overlap is ${\bf x}^{{\bf P}_{k}}\cdot{\bf x}^{{\bf P}'_{k'}}/N=m^{\vert k-k'\vert}$, we perturb the cluster by recorrelating two configurations ${\bf x}^{{\bf P}_{k}}$ and ${\bf x}^{{\bf P}'_{k'}}$: ${\bf x}^{{\bf P}_{k}}\cdot{\bf x}^{{\bf P}'_{k'}}=m^{\vert k-k'\vert-2}$. In App.\ref{['app: simple model FP-gen']}, we show that the stability of the local entropy $s_{\rm loc}(\cdot)$ -with this perturbation- can be mapped to a well-known Franz-Parisi potential computation.
• Figure 5: Plot indicating the sign of the perturbation $\delta s_{\rm loc.}$ as a function of $\alpha$ and of the distance between the two recorrelating configurations ${\bf x}^{{\bf P}_k}$ and ${\bf x}^{{\bf P}'_{k'}}$ in the core of the cluster. The full colored lines indicate the change of sign in the local entropy perturbation for four values of $\kappa$ ($\in\{0.4,0.6,0.8,1\}$). For each value of $\kappa$, we highlighted with a black dot the maximum value of $\alpha$ until which no-memory paths are locally stable everywhere. By this we mean that $\delta s_{\rm loc}$ is negative for all distances. To guide the eye even more, we draw with a dashed green line this critical value of $\alpha$ for $\kappa=0.6$.