Relaxation-assisted reverse annealing on nonnegative/binary matrix factorization

TL;DR

This study proposes an improved strategy that integrates reverse annealing with a linear programming relaxation technique, using relaxed solutions as the initial configuration for reverse annealing, and demonstrates improvements in optimization performance comparable to the exact optimization methods.

Abstract

Quantum annealing has garnered significant attention as meta-heuristics inspired by quantum physics for combinatorial optimization problems. Among its many applications, nonnegative/binary matrix factorization stands out for its complexity and relevance in unsupervised machine learning. The use of reverse annealing, a derivative procedure of quantum annealing to prioritize the search in a vicinity under a given initial state, helps improve its optimization performance in matrix factorization. This study proposes an improved strategy that integrates reverse annealing with a linear programming relaxation technique. Using relaxed solutions as the initial configuration for reverse annealing, we demonstrate improvements in optimization performance comparable to the exact optimization methods. Our experiments on facial image datasets show that our method provides better convergence than known reverse annealing methods. Furthermore, we investigate the effectiveness of relaxation-based initialization methods on randomized datasets, demonstrating a relationship between the relaxed solution and the optimal solution. This research underscores the potential of combining reverse annealing and classical optimization strategies to enhance optimization performance.

Figures (7)

• Figure 1: Squared error at each iteration of the ALS method. The plot shows the squared error at each iteration of the ALS method by using methods shown in Table \ref{['tab:methods']}.
• Figure 2: Squared error over elapsed time. The plot shows the squared error at each iteration of the ALS method by using methods shown in Table \ref{['tab:methods']}.
• Figure 3: Hamming distance from the optimal solution at each iteration of the ALS method. At each iteration, We analyzed the output for the 200 subproblems represented by Equation \ref{['eq:NBMF_H_decomposed']}. The solid line represents the probability density obtained by kernel density estimation.
• Figure 4: Distribution of elements in $H$. We analyzed the matrix $H$ obtained by PGD and the rounding-based method after a single iteration of the ALS method. The histogram represents the distribution of the elements in $H$.
• Figure 5: Example distributions of the randomly generated relaxed solution. Each plot illustrates the distribution of the elements in the relaxed solution randomly generated by our procedure, with shape parameter $\rho$ set to 0.5, 12, 2, and 10. When $\rho$ is small, the solution is concentrated around 0 and 1; as $\rho$ increases, the distribution becomes more Gaussian-like.