Performance Evaluation of Ising and QUBO Variable Encodings in Boltzmann Machine Learning

TL;DR

The paper analyzes how Ising {$-1,+1$} and QUBO {$0,1$} encodings influence Boltzmann Machine learning under a fixed model and sampling setup. By exploiting the identity that the Fisher information matrix $F_ heta$ equals the covariance of sufficient statistics, it compares FIM spectra and spectral entropy across encodings under SGD and NGD. It finds that Ising yields more isotropic curvature and faster convergence under SGD, while QUBO induces cross-statistics coupling leading to ill-conditioning, which NGD helps mitigate. These results provide practical guidelines for encoding choices and preprocessing, with implications for efficient energy-based learning and hardware implementations.

Abstract

We compare Ising ({-1,+1}) and QUBO ({0,1}) encodings for Boltzmann machine learning under a controlled protocol that fixes the model, sampler, and step size. Exploiting the identity that the Fisher information matrix (FIM) equals the covariance of sufficient statistics, we visualize empirical moments from model samples and reveal systematic, representation-dependent differences. QUBO induces larger cross terms between first- and second-order statistics, creating more small-eigenvalue directions in the FIM and lowering spectral entropy. This ill-conditioning explains slower convergence under stochastic gradient descent (SGD). In contrast, natural gradient descent (NGD)-which rescales updates by the FIM metric-achieves similar convergence across encodings due to reparameterization invariance. Practically, for SGD-based training, the Ising encoding provides more isotropic curvature and faster convergence; for QUBO, centering/scaling or NGD-style preconditioning mitigates curvature pathologies. These results clarify how representation shapes information geometry and finite-time learning dynamics in Boltzmann machines and yield actionable guidelines for variable encoding and preprocessing.

Figures (6)

• Figure 1: Comparison on BAS data. Under NGD, convergence matches across encodings; under SGD, Ising reduces KL divergence faster than QUBO.
• Figure 2: Comparison on Ising‑sampled data. NGD yields matching KL convergence; under SGD, Ising converges faster.
• Figure 3: Spectral entropy of the FIM on Ising‑sampled data. Entropy is consistently larger for Ising, indicating a more even curvature allocation.
• Figure 4: Evolution of FIM eigenvalues on Ising‑sampled data ($J_c=1.0$). Ising exhibits larger and more dispersed eigenvalues; QUBO shows a split between large and very small eigenvalues early in training.
• Figure 5: FIM eigenvalues for QUBO (NGD, the 100th iteration). A smaller $J_c$ prolongs the period with extremely small eigenvalues.