How to escape atypical regions in the symmetric binary perceptron: a journey through connected-solutions states

TL;DR

This work probes the geometry of atypical, connected solutions in the binary symmetric perceptron by introducing a solutions chain framework that links consecutive solutions via fixed overlaps. It first proves that planted solutions are isolated, then develops a no-memory Ansatz that yields tractable predictions for delocalized connected states, which Monte Carlo simulations corroborate in finite-size regimes. To access broader regimes, the authors extend the framework with nested Markov memory, showing that memory increases the range of α and κ over which delocalization occurs and capturing non-Markovian correlations observed in simulations. Overall, the paper provides a principled approach to characterize accessible connected solution states and highlights the role of memory and edge-structure in the solution landscape, with implications for algorithmic hardness and search dynamics in high-dimensional constraint satisfaction problems.

Abstract

We study the binary symmetric perceptron model, and in particular its atypical solutions. While the solution-space of this problem is dominated by isolated configurations, it is also solvable for a certain range of constraint density $α$ and threshold $κ$. We provide in this paper a statistical measure probing sequences of solutions, where two consecutive elements shares a strong overlap. After simplifications, we test its predictions by comparing it to Monte-Carlo simulations. We obtain good agreement and show that connected states with a Markovian correlation profile can fully decorrelate from their initialization only for $κ>κ_{\rm no-mem.\, state}$ ($κ_{\rm no-mem.\, state}\sim \sqrt{0.91\log(N)}$ for $α=0.5$ and $N$ being the dimension of the problem). For $κ<κ_{\rm no-mem.\, state}$, we show that decorrelated sequences still exist but have a non-trivial correlations profile. To study this regime we introduce an $Ansatz$ for the correlations that we label as the nested Markov chain.

Figures (12)

• Figure 1: Schematic representation for the geometrical arrangement of the connected solutions we probe in the symmetric binary perceptron. At the center we find the initial planted configuration ${\bf x}_0$. Directly away from it we have the solutions $\bf x_1$, their typical number being $e^{NV_1\left[{\bf x}_0,m,\kappa_1\right]}$. Then, having fixed the solutions ${\bf x}_0$ and $\bf x_1$ we are able to probe $e^{NV_2\left[{\bf x}_0,m,\kappa_1,\kappa_2\right]}$ solutions ${\bf x}_2$. A chain of connected solutions corresponds to a set of solutions $\{{\bf x}_j\}_{j\in [\![1,t]\!]}$ where we have fixed the overlap between to consecutive configurations -${\bf x}_{j+1}\cdot {\bf x}_{j}/N=m$-.
• Figure 2: Sketch of the potential $V_t[{\bf x}_0,m',\{\kappa_{j}\}_{j\in[\![1,t]\!]}]$ as a function of $m'$. In this representation, we focus on a range of overlaps $m'$ close to the chain overlap $m$ (i.e. $1-m'\sim 1-m$). Each color corresponds to a different regime for $\kappa_t$. In red, we have $\kappa_t\leq \kappa_{t-1}$. In this case, the solution ${\bf x}_{t}$ is isolated for $m'\in [m_0,1[$ as the potential is negative. In green, $\kappa_t$ has been slightly increased. Solutions have been created around ${\bf x}_{t-1}$ for a small range of overlaps ($m'\in [m_1,1[$), but an overlap-gap remains (for $m'\in [m_1,m_0[$). The last regime regime, in blue, corresponds to a large increase of $\kappa_t$ -above a critical value $\kappa_c(t)$- for which the overlap-gap disappear. To have this regime at all time in the chain a large and constant increase in $\kappa_t$ is required.
• Figure 3: Representation of the different scenarios for the overlap gap around connected solutions. We have separated the cases $\kappa_t\leq \kappa_{t-1}$ (top) and $\kappa_t> \kappa_{t-1}$ (bottom). For both we have highlighted with a dotted red box the cases for which the chain of solutions stops, i.e when $V_t[{\bf x}_0,m,\{\kappa_{j}\}_{j\in[\![1,t]\!]}]<0$.
• Figure 4: We plot the overlap of the system with its initial configuration ${\bf x}_0$ as it decorrelates via a quench Monte-Carlo dynamics. The quench is performed for $\alpha=0.5$, $\kappa=0.75$ and several sizes of the system $N$. For each size the correlation curve is averaged over 10 realizations of both the dynamics and disorder. It is also attached to a matching color shade which highlight the maximum and minimum overlap values attained over these 10 realizations. On the right, the correlation curves are plotted with the natural time-scale. This means that every time the Monte-Carlo algorithm performs a spin-flip -accepted or not- time is incremented by one. On the left, the same correlation curves are plotted with the rescaled time-scale. In this case, time is incremented by one only when a spin-flip is accepted. In this left panel we added the correlation curve predicted by the no-memory chain. Each colored dot represented the point where the chain stops when setting $m=1-2/N$.
• Figure 5: We plot the overlap of the system with its initial configuration ${\bf x}_0$ as it decorrelates via a quench Monte-Carlo dynamics. The quench is performed for $\alpha=0.5$ and several sizes of the system $N$. For each size the correlation curve is averaged over 10 realizations of both the dynamics and disorder. We set $\kappa$ such that the decorrelation is predicted to stop at exactly ${\bf x}_t \cdot {\bf x}_0/N=0.9$ by the no-memory chain computation. We recall that this prediction depends on the size of the system as we have $m=1-2/N$. On the right, the correlation curves are plotted with the natural time-scale. On the left, the same correlation curves are plotted with the rescaled time-scale. In this left panel we added in dashed black the correlation curve predicted by the no-memory chain.