Beyond-mean-field fluctuations for the solution of constraint satisfaction problems

TL;DR

This work maps MAX-$2$-SAT CSPs to an Ising spin system and analyzes solving by Glauber dynamics, contrasting it with Hopfield networks. A cumulant-based framework reveals how fluctuations, especially spin variances and covariances, enable exploration of degenerate energy plateaus and improve solution quality; mean-field reduces to Hopfield dynamics, while including fluctuations yields improved, sometimes deterministic solvers. At zero temperature, a subset of spins becomes free to flip without energy cost, captured by $m_i$ rules such as $m_i = \operatorname{erf}\left(\frac{\mu_i}{\sqrt{2}\,\sigma_i}\right)$, which underpins enhanced search performance. The resulting variance- and covariance-based solvers achieve state-of-the-art performance on random MAX-$2$-SAT instances, offering a transparent, physics-grounded approach with potential neuromorphic implementations and extensibility to more complex CSPs.

Abstract

Constraint Satisfaction Problems (CSPs) lie at the heart of complexity theory and find application in a plethora of prominent tasks ranging from cryptography to genetics. Classical approaches use Hopfield networks to find approximate solutions while recently, modern machine-learning techniques like graph neural networks have become popular for this task. In this study, we employ the known mapping of MAX-2-SAT, a class of CSPs, to a spin-glass system from statistical physics, and use Glauber dynamics to approximately find its ground state, which corresponds to the optimal solution of the underlying problem. We show that Glauber dynamics outperforms the traditional Hopfield-network approach and can compete with state-of-the-art solvers. A systematic theoretical analysis uncovers the role of stochastic fluctuations in finding CSP solutions: even in the absence of thermal fluctuations at $T=0$ a significant portion of spins, which correspond to the CSP variables, attains an effective spin-dependent non-zero temperature. These spins form a subspace in which the stochastic Glauber dynamics continuously performs flips to eventually find better solutions. This is possible since the energy is degenerate, such that spin flips in this free-spin space do not require energy. Our theoretical analysis leads to deterministic solvers that effectively account for such fluctuations, thereby reaching state-of-the-art performance.

Figures (9)

• Figure 1: Comparison of simulated Glauber dynamics and Hopfield networks.$\textbf{a)}$ Glauber spin dynamics in a random CSP with $\alpha=3$ and $N=400$. The evolution of a subset of $N=100$ spins as a function of time starting from a random initialization. Black indicates a spin in the down state $S_{i}=-1$ while white indicates a spin in the up state $S_{i}=+1$. $\textbf{b)}$ Energy as a function of time for one CSP instance and a single initial condition. $\textbf{c)}$ Fraction of violated clauses dependent on clause density $\alpha=M/N$ averaged over $50$ instances from a single random initial spin configuration. $\textbf{d)}$ Magnetizations of three randomly chosen spins as a function of time $t$ for Hopfield and Glauber dynamics (mean and standard deviation of the mean for $300$ Glauber dynamics realizations) starting from the same initial condition. $\textbf{e)}$ Magnetizations of spins in simulated Glauber dynamics and in the Hopfield network after convergence. All experiments in this figure are done with temperature $T=0.1$. Details for CSP generation are provided in appendix \ref{['app:generation-of-random-csps']}.
• Figure 2: Comparison of mean-field, variance and covariance approximations with simulated Glauber dynamics. Magnetization of two spins as a function of time for simulated Glauber dynamics (mean and standard deviation of the mean over $300$ trials) and solution for mean-field/variance/covariance equations. a,b) Sherrington-Kirkpatrick (SK) model instance at temperatures $T=0.1$ and $T=1$, respectively. c,d) CSP instance at temperatures $T=0.1$ and $T=1$, respectively. The first and second order statistics of the Gaussian couplings $J_{ij}$ and fields $H_{i}$ in the SK instance are matched to the corresponding statistics of the CSP instance. Parameters: $N=400$, $\alpha=3$.
• Figure 3: Analysis of the variance solver.$\textbf{a)}$ Fraction of violated clauses as a function of inverse temperature $1/T$ for the mean-field, variance and covariance (see section \ref{['subsec:Covariance-of-spins:']}) solver averaged over $50$ CSP instances. $\textbf{b)}$ Evolution of spin values over time (white: spin up, black: spin down) for simulated Glauber dynamics at $T=0$ for one example CSP instance with $\alpha=3$ and $100$ randomly chosen spins. $\textbf{c)}$ Number of spin flips as a function of time. The inset additionally shows later times and the black dashed line shows a linear fit for times shortly after initialization. $\textbf{d)}$ Same as panel a) but dependent on spin flips instead of time. $\textbf{e)}$ Simulated and theoretical prediction from the variance solver for variances $\sigma_{i}^{2}$ of spins. Variances of simulated dynamics are measured over $5\cdot10^{4}$ unit times after initialization at theoretical equilibrium state. $\textbf{f)}$ Energy decay for simulated Glauber dynamics as a function of spin flips. Parameters: $N=400$.
• Figure 4: Impact of a small perturbation on Glauber dynamics in a CSP instance.$\textbf{a)}$ Fraction of free spins in a CSP as a function of spin flips for zero temperature. $\textbf{b)}$ Fraction of free spins as a function of clause density. Black line is for original CSP problems, red line for the corresponding perturbed CSPs with additional external field $\epsilon_{i}\sim\mathcal{N}(0,10^{-4})$. The dotted gray line shows the fraction of free spins for an SK-coupling for the original case. The number of free spins was calculated after $10^{4}$ time steps. $\textbf{c)}$ Number of violated clauses as a function of spin flips for the simulated Glauber dynamics for both the original and the perturbed system of the same CSP as in a) at $T=0$. $\textbf{d)}$ Number of violated clauses as a function of the clause density both for the original and the perturbed CSP system at $T=0$, calculated after $10^{4}$ simulation steps. $\textbf{e)}$ Number of violated clauses as a function of inverse temperature for the original and the perturbed system of the CSP from a). Parameters: $N=400$, $\alpha=3$.
• Figure 5: (Co)variance comparison of theory and simulation.a) Left: Frozen and free spins after equilibration as a function of flips at T=0. Free spins are marked in blue. Right: Empirical covariance matrix. Example of $100$ spins. b) Scatter plot of theoretical and simulated covariance. c) Scatter plot of theoretical and simulated variance for both the variance solver (red) and covariance solver (green). Parameters: $N=400$, $\alpha=3$.