Effectiveness of Binary Autoencoders for QUBO-Based Optimization Problems

TL;DR

It is shown that the bAE reconstructs feasible tours accurately and, compared with manually designed encodings at similar compression, better aligns tour distances with latent Hamming distances, better aligns tour distances with latent Hamming distances, yields smoother neighborhoods under small bit flips, and produces fewer local optima.

Abstract

In black-box combinatorial optimization, objective evaluations are often expensive, so high quality solutions must be found under a limited budget. Factorization machine with quantum annealing (FMQA) builds a quadratic surrogate model from evaluated samples and optimizes it on an Ising machine. However, FMQA requires binary decision variables, and for nonbinary structures such as integer permutations, the choice of binary encoding strongly affects search efficiency. If the encoding fails to reflect the original neighborhood structure, small Hamming moves may not correspond to meaningful modifications in the original solution space, and constrained problems can yield many infeasible candidates that waste evaluations. Recent work combines FMQA with a binary autoencoder (bAE) that learns a compact binary latent code from feasible solutions, yet the mechanism behind its performance gains is unclear. Using a small traveling salesman problem as an interpretable testbed, we show that the bAE reconstructs feasible tours accurately and, compared with manually designed encodings at similar compression, better aligns tour distances with latent Hamming distances, yields smoother neighborhoods under small bit flips, and produces fewer local optima. These geometric properties explain why bAE+FMQA improves the approximation ratio faster while maintaining feasibility throughout optimization, and they provide guidance for designing latent representations for black-box optimization.

Figures (5)

• Figure 1: bAE+FMQA workflow for black-box optimization. A binary autoencoder (bAE) trained only on feasible solutions maps non-binary combinatorial solutions to a low-dimensional binary latent code. In the latent space, a factorization machine (FM) is trained on previously evaluated samples and their objective values, and its quadratic approximation is formulated as a QUBO and optimized using an Ising machine (e.g., quantum annealing). The optimized latent code is decoded back to the original solution representation and evaluated, and the resulting sample is added to the dataset for the next iteration, enabling black-box optimization without relying on handcrafted encodings.
• Figure 2: Training curves of the bAE with $(d_z,d_h)=(14,64)$ on feasible tours. (a) Reconstruction loss $\mathcal{L}_{\mathrm{MSE}}$ as a function of epoch. (b) Reconstruction accuracy $\mathcal{A}$ as a function of epoch. Both training and validation metrics are shown.
• Figure 3: Dependence of reconstruction accuracy on the latent dimension $d_z$ and the hidden-layer size $d_h$. (a) Reconstruction accuracy $\mathcal{A}$ versus $d_z$ with $d_h=64$. (b) Reconstruction accuracy $\mathcal{A}$ versus $d_h$ with $d_z=14$. Each point shows the mean over five independent training runs with different random seeds. Error bars indicate the standard deviation. Lines connecting the points are drawn as a guide to the eye.
• Figure 4: Comparison of structure preservation in the binary space for the bAE, rank-based log/gray encoding, and random label encoding. (a) Spearman rank correlation coefficient between the edge distance in the original tour space and the Hamming distance in the binary space. (b) Neighborhood distance characteristic $L(m)$, which measures the change in tour distance induced by flipping $m$ bits in the binary space. (c) Local optimum ratio $r_{\mathrm{Local}}$, i.e., the fraction of binary solutions that are local optima under single-bit flips.
• Figure 5: Optimization performance of bAE+FMQA. (a) Approximation ratio $R$ as a function of FMQA iterations, where $R\approx1$ indicates reaching the exact optimum. (b) Feasible-sample probability $P_{\mathrm{Feasible}}$, defined as the fraction of raw Ising machine outputs that satisfy the constraints before any post-processing. Error bars indicate mean $\pm$ standard deviation over problem instances.