SWIFT-FMQA: Enhancing Factorization Machine with Quadratic-Optimization Annealing via Sliding Window

TL;DR

This work tackles the stagnation observed in black-box optimization with FMQA by addressing data dilution in surrogate training. It introduces SWIFT-FMQA, a sliding-window extension that caps the training dataset to the most recent $D_{latest}$ data points, preserving the influence of new, informative samples while maintaining generalization. Empirical results on the LABS benchmark show SWIFT-FMQA consistently finds lower-cost solutions with fewer BB evaluations than FMQA, particularly when $D_{latest}$ is around 100, and demonstrates robustness to the FM hyperparameter $K$ and initial data size $D_{init}$. The approach offers a practical, data-efficient improvement for FMQA and related Ising-machine-assisted BB optimization tasks, with potential applicability to real-world, resource-limited design problems.

Abstract

Black-box (BB) optimization problems aim to identify an input that maximizes or minimizes the output of a function (the BB function) whose input-output relationship is unknown. Factorization machine with quadratic-optimization annealing (FMQA) is a promising approach to this task, employing a factorization machine (FM) as a surrogate model to iteratively guide the solution search via an Ising machine. Although FMQA has demonstrated strong optimization performance across various applications, its performance often stagnates as the number of optimization iterations increases. One contributing factor to this stagnation is the growing number of data points in the dataset used to train FM. As more data are accumulated, the contribution of newly added data points tends to become diluted within the entire dataset. Based on this observation, we hypothesize that such dilution reduces the impact of new data on improving the prediction accuracy of FM. To address this issue, we propose a novel method named sliding window for iterative factorization training combined with FMQA (SWIFT-FMQA). This method improves upon FMQA by utilizing a sliding-window strategy to sequentially construct a dataset that retains at most a specified number of the most recently added data points. SWIFT-FMQA is designed to enhance the influence of newly added data points on the surrogate model. Numerical experiments demonstrate that SWIFT-FMQA obtains lower-cost solutions with fewer BB function evaluations compared to FMQA.

Figures (8)

• Figure 1: Illustration of the flow of FMQA, including SWIFT-FMQA. Unlike FMQA, SWIFT-FMQA sets an upper limit on the number of data points in the dataset; therefore, some existing data points are discarded during the dataset construction.
• Figure 2: Comparison in terms of the dataset construction between (a) FMQA and (b) SWIFT-FMQA. The plus marks represent $D_{\mathrm{adds}}$ solutions added to the dataset in one optimization iteration of FMQA and SWIFT-FMQA. The total number of optimization iterations is $N_{\mathrm{iter}}$. $D_{\mathrm{latest}}$ represents the upper bound on the number of data points in the dataset in SWIFT-FMQA. In this illustration, $D_{\mathrm{adds}}=2$ and $D_{\mathrm{latest}}=3$.
• Figure 3: Analysis of the residual value with respect to (a) $D_{\mathrm{latest}}$ and (b) optimization iterations. The markers and lines indicate the mean residual values obtained from $50$ simulations, while the error bars and shaded areas represent their standard deviation. (a) Dependence of the lowest residual value on $D_{\mathrm{latest}}$, evaluated after $N_{\mathrm{iter}}$ optimization iterations with SWIFT-FMQA completed. For comparison, the lowest residual value obtained with FMQA is also shown. (b) Transition of the lowest residual value over the optimization iterations. For visual clarity, markers are displayed only for the representative cases of SWIFT-FMQA with $D_{\mathrm{latest}}=100$ and FMQA.
• Figure 4: Parameter dependence of the improvement rate for $D_{\mathrm{latest}} = 100$. (a) Dependence on the problem size $N$. (b) Dependence on the FM hyperparameter $K$. (c) Dependence on the number of data points used for initial training, $D_{\mathrm{init}}$. For panels (b) and (c), the problem size is fixed at $N = 64$. All parameters other than the one varied along the horizontal axis are set according to Table \ref{['table:Parameter_for_FMQA']}.
• Figure 5: $D_{\mathrm{latest}}$ dependency of the obtained lowest residual value until the $N_{\mathrm{iter}}$ optimization iterations have been completed. The error bars indicate the standard deviation of the residual values obtained from $50$ simulations. (a) $N=16$. (b) $N=49$. (c) $N=81$. (d) $N=101$.