Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boltzmann Reinforcement Learning for Noise resilience in Analog Ising Machines

Aditya Choudhary, Saaketh Desai, Prasad Iyer

TL;DR

This work addresses noise-limited sampling and optimization on Analog Ising Machines by reframing the problem as distribution learning. It introduces BRAIN, a variational reinforcement approach that learns a factorized Bernoulli sampler $q_\theta(x)$ to approximate the Boltzmann distribution $p(x) = \frac{e^{-\beta E(x)}}{Z}$ using noisy energy measurements. By applying REINFORCE with baselines and keeping the sampler lightweight, BRAIN achieves high ground-state fidelity under realistic noise (e.g., 3% relative energy noise) and significantly accelerates solution time compared with MCMC, while scaling to tens of thousands of spins ($\mathcal{O}(N^{1.55})$). The method also captures thermodynamic phase transitions and metastable states across topologies, including Curie–Weiss and Lenz–Ising models, demonstrating robustness to measurement uncertainty up to 40%. This approach holds promise for enabling robust, low-latency optimization on hardware that inherently exhibits stochastic energy evaluations.

Abstract

Analog Ising machines (AIMs) have emerged as a promising paradigm for combinatorial optimization, utilizing physical dynamics to solve Ising problems with high energy efficiency. However, the performance of traditional optimization and sampling algorithms on these platforms is often limited by inherent measurement noise. We introduce BRAIN (Boltzmann Reinforcement for Analog Ising Networks), a distribution learning framework that utilizes variational reinforcement learning to approximate the Boltzmann distribution. By shifting from state-by-state sampling to aggregating information across multiple noisy measurements, BRAIN is resilient to Gaussian noise characteristic of AIMs. We evaluate BRAIN across diverse combinatorial topologies, including the Curie-Weiss and 2D nearest-neighbor Ising systems. We find that under realistic 3\% Gaussian measurement noise, BRAIN maintains 98\% ground state fidelity, whereas Markov Chain Monte Carlo (MCMC) methods degrade to 51\% fidelity. Furthermore, BRAIN reaches the MCMC-equivalent solution up to 192x faster under these conditions. BRAIN exhibits $\mathcal{O}(N^{1.55})$ scaling up to 65,536 spins and maintains robustness against severe measurement uncertainty up to 40\%. Beyond ground state optimization, BRAIN accurately captures thermodynamic phase transitions and metastable states, providing a scalable and noise-resilient method for utilizing analog computing architectures in complex optimizations.

Boltzmann Reinforcement Learning for Noise resilience in Analog Ising Machines

TL;DR

This work addresses noise-limited sampling and optimization on Analog Ising Machines by reframing the problem as distribution learning. It introduces BRAIN, a variational reinforcement approach that learns a factorized Bernoulli sampler to approximate the Boltzmann distribution using noisy energy measurements. By applying REINFORCE with baselines and keeping the sampler lightweight, BRAIN achieves high ground-state fidelity under realistic noise (e.g., 3% relative energy noise) and significantly accelerates solution time compared with MCMC, while scaling to tens of thousands of spins (). The method also captures thermodynamic phase transitions and metastable states across topologies, including Curie–Weiss and Lenz–Ising models, demonstrating robustness to measurement uncertainty up to 40%. This approach holds promise for enabling robust, low-latency optimization on hardware that inherently exhibits stochastic energy evaluations.

Abstract

Analog Ising machines (AIMs) have emerged as a promising paradigm for combinatorial optimization, utilizing physical dynamics to solve Ising problems with high energy efficiency. However, the performance of traditional optimization and sampling algorithms on these platforms is often limited by inherent measurement noise. We introduce BRAIN (Boltzmann Reinforcement for Analog Ising Networks), a distribution learning framework that utilizes variational reinforcement learning to approximate the Boltzmann distribution. By shifting from state-by-state sampling to aggregating information across multiple noisy measurements, BRAIN is resilient to Gaussian noise characteristic of AIMs. We evaluate BRAIN across diverse combinatorial topologies, including the Curie-Weiss and 2D nearest-neighbor Ising systems. We find that under realistic 3\% Gaussian measurement noise, BRAIN maintains 98\% ground state fidelity, whereas Markov Chain Monte Carlo (MCMC) methods degrade to 51\% fidelity. Furthermore, BRAIN reaches the MCMC-equivalent solution up to 192x faster under these conditions. BRAIN exhibits scaling up to 65,536 spins and maintains robustness against severe measurement uncertainty up to 40\%. Beyond ground state optimization, BRAIN accurately captures thermodynamic phase transitions and metastable states, providing a scalable and noise-resilient method for utilizing analog computing architectures in complex optimizations.
Paper Structure (28 sections, 1 theorem, 23 equations, 11 figures, 2 tables, 1 algorithm)

This paper contains 28 sections, 1 theorem, 23 equations, 11 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Description
  3. Related Work
  4. Combinatorial Optimization: Ground State Finding
  5. Distribution Learning: Sampling the Boltzmann Distribution
  6. Algorithms for Noisy Analog Ising Systems
  7. Methods
  8. Experiments
  9. Benchmarking on Low-Dimensional Energy Landscapes
  10. Evaluating BRAIN on High-Dimensional Analog Ising Networks
  11. Noisy Curie-Weiss Ising Hamiltonian
  12. Lenz–Ising model: Generalization Beyond Mean-Field Ising Networks
  13. Conclusions and Outlook
  14. Appendixes
  15. Characterizing the noise in the analog Ising machine
  16. ...and 13 more sections

Key Result

Proposition A.1

Let $g^{(s)}(\theta)$ be the gradient estimator with batch size $s$, and $g^{(s,b)}(\theta)$ be the estimator with baseline subtraction $b = \frac{1}{s}\sum_{k=1}^s \tilde{E}(x_k)$. Under independent multiplicative Gaussian noise $\eta \sim \mathcal{N}(0, \sigma^2)$, the difference in noise-induced where $\overline{E^2} =\frac{1}{s}\sum_{k=1}^s E(x_k)^2$, $a_k \equiv \nabla_\theta \log q_\theta(x

Figures (11)

  • Figure 1: (a) A noisy double-well energy landscape $E(x)$, and associated probability $p(x)$ of observing state $x$. (b) Comparing BRAIN and MCMC to the ground truth $p(x)$ for two different temperatures. We find that both MCMC and brain perform adequately at high temperature but MCMC performs worse at low temperatures. (c) The energy landscape $E(x)$ and associated probability $p(x)$ for a one-dimensional six spin system. (d) Comparing BRAIN and MCMC to ground truth at a single, low temperature - we find both algorithms to be equally adequate at representing $p(x)$.
  • Figure 2: (a) All-to-all coupled Ising network representing the Curie-Weiss Hamiltonian where every spin on the network is coupled to every other spin independent of the distance between the spins. Temperature-dependent magnetization profile for (b) MCMC and (c) BRAIN under various noise levels. The inset in (b) and (c) represents the ground-state spin state at T = 0.33 with 3% noise. (d) MCMC and BRAIN convergence at T=0.33 under 3% noise, shown in terms of the number of energy evaluations required to reach optimal solutions. (e) Minimum number of samples needed for BRAIN to maintain solution fidelity as noise increases. (f) Scalability analysis across system sizes (N) ranging from 1,024 to 65,536 spins. The left axis displays the number of energy evaluations at T=0.33 under both noiseless and 3% noise conditions, while the right axis shows the corresponding magnetization values.
  • Figure 3: (a) Schematic representing a 2D square array of spins emulating the classic Ising Hamiltonian with nearest-neighbor interactions (arrows). Temperature-dependent magnetization, comparing MCMC (b) and BRAIN (c) under increasingly noisy energy evaluations.
  • Figure 4: (a) Fluctuation in energy observed for a fixed spin configuration using the Spatial Photonic Ising Machine (SPIM) across 500 experimental trials. (b) Histogram illustrating the distribution of measured energy values.
  • Figure 5: Magnetization as a function of temperature for the parallel tempering algorithm, comparing performance under noiseless (0% noise) and noisy (3%) energy evaluations.
  • ...and 6 more figures

Theorems & Definitions (2)

  • Proposition A.1
  • proof