Lagrange Oscillatory Neural Networks for Constraint Satisfaction and Optimization

TL;DR

LagONN introduces a deterministic, physics-inspired approach to constraint satisfaction by augmenting oscillatory neural networks with Lagrange oscillators that enforce clause-level constraints for Max-3-SAT. By defining clause-specific energies $Z_i$ and a Lagrange function $L_i(\phi,\phi_\lambda)$, the network performs gradient descent in the phase variables while the Lagrange oscillator performs gradient ascent to drive all $Z_i$ to zero, effectively locating a feasible, optimal assignment. The authors present a modular architecture, showing that high-order interactions can be implemented locally within clause modules, and demonstrate competitive performance against simulated annealing and SAT solvers up to 200 variables and 860 clauses, with clear regimes where LagONN excels by avoiding infeasible minima and eliminating the need for annealing. The work suggests broader applicability to other constrained-optimization problems, including phase copying, and points to practical hardware considerations such as SHIL binarization and stability management for scalable implementations.

Abstract

Physics-inspired computing paradigms are receiving renewed attention to enhance efficiency in compute-intensive tasks such as artificial intelligence and optimization. Similar to Hopfield neural networks, oscillatory neural networks (ONNs) minimize an Ising energy function that embeds the solutions of hard combinatorial optimization problems. Despite their success in solving unconstrained optimization problems, Ising machines still face challenges with constrained problems as they can become trapped in infeasible local minima. In this paper, we introduce a Lagrange ONN (LagONN) designed to escape infeasible states based on the theory of Lagrange multipliers. Unlike existing oscillatory Ising machines, LagONN employs additional Lagrange oscillators to guide the system towards feasible states in an augmented energy landscape, settling only when constraints are met. Taking the maximum satisfiability problem with three literals as a use case (Max-3-SAT), we harness LagONN's constraint satisfaction mechanism to find optimal solutions for random SATlib instances with up to 200 variables and 860 clauses, which provides a deterministic alternative to simulated annealing for coupled oscillators. We benchmark LagONN with SAT solvers and further discuss the potential of Lagrange oscillators to address other constraints, such as phase copying, which is useful in oscillatory Ising machines with limited connectivity.

Figures (11)

• Figure 1: a) Any combinatorial optimization problem can be mapped to the 3-SAT problem Cook_1971. Its optimization version Max-3-SAT seeks an assignment $X$ satisfying most clauses and can be described as the minimization of an Ising energy function $H(X)$. b) Phase-based oscillatory neural networks (ONN) are analog solvers for combinatorial optimization. Their energy function $E(\phi)$ corresponds to a continuous relaxation of $H(X)$. ONNs can be trapped in local energy minima. c) The proposed Lagrange ONN employs additional oscillators to enforce constraint satisfaction corresponding to 3-SAT clauses. Conceptually, these forces correspond to vectors $\Vec{u_\lambda}$ that "push" an energy vector $\Vec{Z}$ along new directions to escape local minima and reach an optimal solution.
• Figure 2: a) 3-SAT clause mapping to ONN. The Ising energy is relaxed to a complex quantity $Z_i$, a function of ONN phases $\phi_X,\phi_Y,\phi_Z$. For binary phases, $Z_i=0$ corresponds to the optimal Ising state and induces $C_i=$ TRUE. However, the standard ONN trajectory settles to an undesired fixed point where $Z_i\neq 0$ (bottom). b) Adding a Lagrange oscillator with phase $\phi_\lambda$ can enforce the constraint $Z_i=0$. This is achieved by defining the Lagrange function $L_i$ and setting competitive dynamics (gradient descent and ascent) seeking a saddle point of $L_i$ where $Z_i=0$. Bottom: resulting Lagrange ONN trajectory for the same phase initialization.
• Figure 3: a) LagONN network for solving clause $C_1$. Rectangles denote complex synapses in the form $J_{ij}\exp{(i\theta_{ij}})$ where $\theta_{ij}$ is implemented with a delay $\theta_{ij}/\omega_0$ in practice. The connections for $\theta_{ij}$ are not shown. b) Modular LagONN architecture to program any 3-SAT formula $f_B$ with $N$ variables and $M$ clauses. Each rectangle corresponds to a clause subcircuit and contains a Lagrange oscillator. The clause selection signal programs the clause subcircuit to one of the four possible clauses. $N\times1$ multiplexers route oscillator input and output signals to each clause according to the desired Boolean formula $f_B$.
• Figure 4: Phase dynamics for the satisfiable SATlib instance 'cnf-20-01' with $N=20$ variables and $M=91$ clauses. a) Standard ONN dynamics quickly converge toward a sub-optimal solution where 89/91 of clauses are satisfied. b) With the same initialization, the Lagrange version takes more time to reach a fixed point. Reading out the phases gives an optimal Boolean assignment where all clauses are satisfied. c) Cost function comparison between the two approaches. LagONN finds an assignment of optimal phase around the same time the standard ONN settles ($\approx$ 10 oscillation cycles). By measuring the cost in real time, we can stop the run when a target cost is reached without waiting for convergence. d) Dynamics for four LagONN energy terms $Z_m$ corresponding to the first four clauses. While the dynamics almost seem chaotic, they evolve to reach a target saddle point where all $Z_m=0$. Ultimately, the dynamics converge toward a fixed point at t=300 oscillation cycles where all $Z_m=0$.
• Figure 5: a) Cost evolution comparison for a SATlib instance with 100 variables and 430 clauses, where all variables are initialized to '1'. The temperature in simulated annealing decreases exponentially with the number of steps as described in the Appendix \ref{['appendix_SA']}. b) Estimation of the time to reach an optimal solution (TTS) with 99% probability for SATlib instances with a clauses-to-variables ratio $M/N\approx 4.3$. Boxes show the 1st, 2nd, and 3rd quartiles computed for the first 30 satisfiable instances from SATlib for LagONN, and 100 instances for simulated annealing. The error bars show the min-max values, and red circles are the averages. For both methods, we fit the logarithm of data to estimate the mean TTS scaling as $\sim \exp(aN+b)$ with $a=0.059$, $b=3.58$ for LagONN and $a=0.055$, $b=8.16$ for simulated annealing (red dashed lines). Simulated annealing has an advantageous scaling for the median TTS fitted as $\sim \exp(c\sqrt{N}+d)$ with $c=0.719$, $d=5.49$, whereas we fit LagONN's median TTS as $\sim \exp(aN+b)$ with $a=0.056$ and $b=2.43$ (blue dashed curves). c) For a fixed number of variables $N=50$, we vary the number of clauses $M$ from 100 to 218 and compute TTS for 100 instances per point. The median TTS scales exponentially with the number of clauses for both methods. d) Benchmark of median TTS with SAT solvers from Ref. Pedretti_2025 based on stochastic local search for satisfiable SATlib instances.

Theorems & Definitions (4)

• Theorem 1
• Theorem 2
• Theorem 2
• proof