Accelerating Continuous Variable Coherent Ising Machines via Momentum

TL;DR

Ising-like optimization with continuous variables can benefit from analog hardware such as the Coherent Ising Machine (CIM). We replace the standard gradient-descent feedback in CV-CIM with momentum and Adam updates, analyzing a Gaussian-state CIM model and benchmarking on BoxQP instances. The results show that momentum-CV-CIM and Adam-CV-CIM speed up convergence and yield more diverse samples than the original GD-CV-CIM, with Adam-CV-CIM also displaying robustness to feedback strength and problem conditioning. This work demonstrates a productive interface between classical optimization techniques and non-conventional analog computing hardware, pointing to future directions in feedback design and continuous-time modeling of CIMs.

Abstract

The Coherent Ising Machine (CIM) is a non-conventional architecture that takes inspiration from physical annealing processes to solve Ising problems heuristically. Its dynamics are naturally continuous and described by a set of ordinary differential equations that have been proven to be useful for the optimization of continuous variables non-convex quadratic optimization problems. The dynamics of such Continuous Variable CIMs (CV-CIM) encourage optimization via optical pulses whose amplitudes are determined by the negative gradient of the objective; however, standard gradient descent is known to be trapped by local minima and hampered by poor problem conditioning. In this work, we propose to modify the CV-CIM dynamics using more sophisticated pulse injections based on tried-and-true optimization techniques such as momentum and Adam. Through numerical experiments, we show that the momentum and Adam updates can significantly speed up the CV-CIM's convergence and improve sample diversity over the original CV-CIM dynamics. We also find that the Adam-CV-CIM's performance is more stable as a function of feedback strength, especially on poorly conditioned instances, resulting in an algorithm that is more robust, reliable, and easily tunable. More broadly, we identify the CIM dynamical framework as a fertile opportunity for exploring the intersection of classical optimization and modern analog computing.

Figures (6)

• Figure 1: This figure shows how the roundtrip ratio is computed for a single problem instance. We determine the best relative optimality gap achieved by the 5th percentile of both CIM variants (8e-3 at the end of GD-CV-CIM's evolution) and compare the roundtrip numbers where the 5th percentile first reaches this gap ($\sim$ 9500 for momentum, 30000 for gradient descent). The roundtrip ratio is approximately 0.32, and we indicate which variant achieves the target gap faster.
• Figure 2: This figure plots the histogram of roundtrip fractions, computed according to the process illustrated in Figure \ref{['fig:time_frac_ex']}. The top row compares the Adam-CV-CIM to the GD-CV-CIM, while the bottom row compares the momentum-CV-CIM to the GD-CV-CIM. The data points are categorized depending on which CIM feedback variant converged faster.
• Figure 3: This figure plots the relative optimality gaps for three representative instances. In all instances, the Adam-CV-CIM and momentum-CV-CIM achieve a greater diversity of samples with multi-modal distributions. For spar040-040-3, all samples from GD-CV-CIM concentrate on the global optimum, while for spar080-050-3, all samples concentrate on a suboptimal solution. For spar125-025-1, all variants result in multi-modal distributions with some shared modes between solvers.
• Figure 4: This figure plots the histogram of optimality gap standard deviations for all CIM variants. The momentum-CV-CIM has the highest standard deviations, with values roughly double that of the Adam-CV-CIM. The GD-CV-CIM has very low variation in optimality gaps, with most instances having no variation.
• Figure 5: This plot shows the likelihood of samples having a relative optimality gap $\leq 0.1 \%$. Adam-CV-CIM and momentum-CV-CIM consistently achieved this, while GD-CV-CIM often met the target, except in 17 instances out of 99.