The geometry and dynamics of annealed optimization in the coherent Ising machine with hidden and planted solutions

TL;DR

This work analyzes how annealing shapes the high-dimensional energy landscape of the coherent Ising machine (CIM) when solving problems with hidden planted solutions. By combining replica theory, random-matrix theory, Kac–Rice methods, and dynamical mean-field theory, it maps phase diagrams for global minima and most abundant local minima in SK-like and Wishart planted ensembles, highlighting soft modes that enable energy descent. The authors show that annealed CIM dynamics can exploit soft eigenmodes near near-global minima to reach lower energies than spectral methods, but experience a rigidity transition beyond which further annealing yields diminishing returns. They also connect supersymmetry-breaking order parameters to grand-potential susceptibilities, and demonstrate through DMFT and simulations that typical CIM trajectories align with low-energy global minima rather than abundant high-energy local minima. Overall, the work reveals deep links between high-dimensional geometry and the dynamics of analog optimization devices, with implications for designing more effective CIM architectures and annealing protocols.

Abstract

The coherent Ising machine (CIM) is a nonconventional hardware architecture for finding approximate solutions to large-scale combinatorial optimization problems. It operates by annealing a laser gain parameter to adiabatically deform a high-dimensional energy landscape over a set of soft spins, going from a simple convex landscape to the more complex optimization landscape of interest. We address how the evolving energy landscapes guides the optimization dynamics against problems with hidden planted solutions. We study the Sherrington-Kirkpatrick spin-glass with ferromagnetic couplings that favor a hidden configuration by combining the replica method, random matrix theory, the Kac-Rice method and dynamical mean field theory. We characterize energy, number, location, and Hessian eigenspectra of global minima, local minima, and critical points as the landscape evolves. We find that low energy global minima develop soft-modes which the optimization dynamics can exploit to descend the energy landscape. Even when these global minima are aligned to the hidden configuration, there can be exponentially many higher energy local minima that are all unaligned with the hidden solution. Nevertheless, the annealed optimization dynamics can evade this cloud of unaligned high energy local minima and descend near to aligned lower energy global minima. Eventually, as the landscape is further annealed, these global minima become rigid, terminating any further optimization gains from annealing. We further consider a second optimization problem, the Wishart planted ensemble, which contains a hidden planted solution in a landscape with tunable ruggedness. We describe CIM phase transitions between recoverability and non-recoverability of the hidden solution. Overall, we find intriguing relations between high-dimensional geometry and dynamics in analog machines for combinatorial optimization.

Figures (14)

• Figure 1: Schematic phase diagram for global minima and most abundant local minima of the coherent Ising machine. In each phase, we summarize the properties of global (black curves) and local (grey curves) minima by their positions in a three-dimensional coordinate system indicating their typical energy $E$, square distance from the origin $q$, and the net magnetization $m$. Additionally, we indicate whether either minima are soft (wide dashed curves) or rigid (sharp solid curves). At low laser gain and ferromagnetic coupling (Sec. \ref{['sec:Hessian_origin']}), the system is in a paramagnetic phase, with a single rigid global minimum located at the origin (light green region). If the ferromagnetic coupling is large enough, further increase of the laser gain leads to a transition of the Baik-Ben Arous Peché type toward a ferromagnetic phase (dark green region, Sec. \ref{['sec:phase_boundary_para_to_ferro']}). As the laser gain increases further, both the paramagnetic and the ferromagnetic phases become unstable toward a spin-glass (SG) phase (yellow and red regions). This overall SG phase can be subdivided into two phases, according to the behavior of the spectrum of the Hessian at global minima (analyzed in Sec. \ref{['sec:hessian_spin_glass']}). For intermediate values of laser gain and ferromagnetic couplings, the lower edge of the Hessian at global minima touches the origin yielding a 'soft' spin-glass (light paramagnetic and dark ferromagnetic yellow regions). At larger values of the laser gain or ferromagnetic coupling, the spectrum of the Hessian at global minima is gapped away from the origin, yielding a rigid spin-glass (light paramagnetic and dark ferromagnetic red regions). Using a replicated Kac-Rice calculation (Sec \ref{['sec:Kac-Rice']}), we further add to this picture by studying the properties of the most abundant local minima across the phase diagram. This yields a further subdivision of the soft spin-glass phase by a critical line (gray dash-dotted line) into two sub-phases. Above the critical line (but not below), local minima proliferate at an exponential rate with the size of the system. Interestingly, these minima are paramagnetic for any value of the ferromagnetic coupling, unlike the global minima, exist at higher energy than the global minima, and they are soft, with a gapless Hessian eigenspectrum. Furthermore, these exponentially many higher energy paramagnetic local minima persist into rigid spin-glass phase, and they retain their soft character there, despite the fact that global minima become rigid (light and dark red regions) Numerical simulations and dynamical mean field theory (Sec. \ref{['sec:dynamics']}) demonstrate that the annealed optimization dynamics of the CIM visits spin configurations that are much more similar to global minima than to the most abundant local minima. Importantly, annealing across the soft SG phase allows the CIM to evade higher energy local minima and reach better approximate solutions to the original combinatorial optimization problem, by exploiting soft modes of near global minima to descend. But when the annealed optimization dynamics enters the rigid SG phase, the dynamics finally terminates and the energy of the solution to the combinatorial optimization problem does not decrease further.
• Figure 2: Phase diagram of global minima of the CIM-SK with ferromagnetic bias Panel (a): phase diagram in the $(J_0,\,a)$ plane: we identify a paramagnetic convex phase I, a ferromagnetic phase II, a spin-glass phase III endowed with soft modes, and a rigid spin-glass phase IV. The color map in the plot is the mean value of the single spin distribution $P^\text{\tiny{SK}}(x)$ at global minima. The black lines identify different phase boundaries in the plane $(a,J_0)$. The dashed line $a_\text{\tiny{BBP}}(J_0)$ denotes the boundary between the paramagnetic phase and the ferromagnetic phase. The dotted line $a_\text{\tiny{sg}}(J_0)$ is the boundary between the replica symmetric phases and the spin-glass phase with soft modes at the global minima, and the solid line $a_\text{\tiny{r}}(J_0)$ is the boundary for the rigid spin-glass phase. Panels (b-e): single-spin distribution $P^\text{\tiny{SK}}(x)$ and distribution of eigenvalues of the Hessian $\rho^\text{\tiny{SK}}(\lambda)$ (inset) at global minima in the different phases. The blue lines are the analytical results, while the orange histograms are lowest energy minima sampled from small, finite size systems. Panel (b): in the convex paramagnetic phase the soft spins are localized at the origin and the spectral distribution follows the Wigner semicircle law. Panel (c): in the ferromagnetic phase the distribution of the spin is skewed, with a finite probability density at the origin, and the spectrum of the Hessian is gapped. Panel (d): in the non-rigid spin-glass phase the single spin distribution is bimodal, with a nonzero probability density at the origin. The spectrum of the Hessian is gapless. Panel (e): in the rigid spin-glass phase, the single-spin probability density is nonzero in two disconnected domains along the $x$-axis, and the spectrum of the Hessian is gapped.
• Figure 3: Lower edge of the spectrum of the Hessian of global minima in the annealed landscape. We plot $\lambda_\text{\tiny{min}}^\text{\tiny{SK}}(J_0,a)$ as a function of the laser gain $a$ for two different values of the mean connectivity $J_0$. Panel (a): in the replica-symmetric phase, the lower edge of the spectrum decreases as the laser gain increases, until it hits zero (horizontal solid red line) at the spin-glass transition value $a_{\mathrm{sg}}(J_0)$ (vertical dotted black line). Upon further increase of the laser gain in the spin-glass phase, the lower edge of the Hessian remains zero, as long as the laser gain is below the rigidity transition value $a_\mathrm{r}$ (vertical solid black line). For $a>a_\mathrm{r}$, the spectrum is gapped. In panel (b), the spin-glass and the rigidity transition occur at the same value of the laser gain, $a_\text{\tiny{r}}=a_\text{\tiny{sg}}$. The spectrum has a gap both above and below this value.
• Figure 4: Proliferation of paramagnetic critical points Phase diagram of the most abundant minima and critical points of the CIM-SK, as obtained from the replicated Kac-Rice calculation. The heat map is the mean of the soft spin variable $\langle x \rangle_\Omega^\text{\tiny{SK}}$, obtained for $\mu\to -\infty$. The dash-dotted line identifies a critical value $a_\Sigma(J_0)$ which separates a region where the complexity is zero from a region with an exponentially large (in the size of the system) number of paramagnetic local minima and critical points. In this region, the index of the most abundant critical points, $r^*$, is larger than $0$, i.e., the most abundant critical points are saddles. The mean value of the soft spin distribution is zero both for the most abundant minima and for the most abundant critical points.
• Figure 5: The most abundant critical points are paramagnetic. Scatter plots of critical points sampled from systems of small size $N=12$, at two different values of the ferromagnetic bias $J_0$, for values of the laser gain $a$ such that the ground state of the system is in a spin-glass. The green line is the prediction from the Kac-Rice theory, the color of the points is the intensive index $r$ of the critical points, and the black dots are the properties of the most abundant critical points at fixed values of the intensive index $r$. Panels (a-b): scatter plots of the intensive energy as a function of the self-overlap $q_d \equiv \frac{1}{N} \sum_i \langle x_i^2\rangle_\text{\bf J}$ (mean squared distance of a soft-spin configuration from the origin), and of the intensive energy as a function of the mean of the soft spin distribution $m\equiv \frac{1}{N}\sum_i \langle x_i\rangle_{\text{\bf J}}$ for $J_0=0$. Panels (c-d): plots of the same quantities, but obtained for a system with large ferromagnetic coupling $J_0$.