Non-Negative Matrix Factorization Using Non-Von Neumann Computers

TL;DR

This paper investigates solving non-negative matrix factorization (NMF) on non-von Neumann, energy-based architectures, focusing on the Dirac-3 entropy computer. It presents two formulations: QuarDP, a quartic objective for real/integer variables, and a QUBO-based approach with binary encodings; both mapped to Dirac-3 and evaluated against Scikit-learn's NMF and Google's CP-SAT. Key findings include that QuarDP under Dirac-3 is outperformed by conventional solvers on small real-valued problems, but a fusion strategy using Dirac-3 outputs as initializations can improve reconstruction error; for integer NMF, serial CP-SAT generally outperforms Dirac-3 while parallel CP-SAT gains advantage in some cases. The results highlight potential domains where NVN entropy computing could offer advantages and outline directions for scalable, energy-efficient hybrid methods and all-optical implementations.

Abstract

Non-negative matrix factorization (NMF) is a matrix decomposition problem with applications in unsupervised learning. The general form of this problem (along with many of its variants) is NP-hard in nature. In our work, we explore how this problem could be solved with an energy-based optimization method suitable for certain machines with non-von Neumann architectures. We used the Dirac-3, a device based on the entropy computing paradigm and made by Quantum Computing Inc., to evaluate our approach. Our formulations consist of (i) a quadratic unconstrained binary optimization model (QUBO, suitable for Ising machines) and a quartic formulation that allows for real-valued and integer variables (suitable for machines like the Dirac-3). Although current devices cannot solve large NMF problems, the results of our preliminary experiments are promising enough to warrant further research. For non-negative real matrices, we observed that a fusion approach of first using Dirac-3 and then feeding its results as the initial factor matrices to Scikit-learn's NMF procedure outperforms Scikit-learn's NMF procedure on its own, with default parameters in terms of the error in the reconstructed matrices. For our experiments on non-negative integer matrices, we compared the Dirac-3 device to Google's CP-SAT solver (inside the Or-Tools package) and found that for serial processing, Dirac-3 outperforms CP-SAT in a majority of the cases. We believe that future work in this area might be able to identify domains and variants of the problem where entropy computing (and other non-von Neumann architectures) could offer a clear advantage.

Figures (5)

• Figure 1: (a) Median relative errors $\delta_{V}$ of the reconstructed matrices $\tilde{V}$ for the test cases in experiment I (with respect to their size) and (b) Median runtime in seconds for the same (for one run). The Dirac-3 results correspond to their relaxation schedules (RS) for which they were run, and sklearn refers to the results from Scikit-learn. The dashed lines for the results of Dirac-3 are curves fitted to the corresponding medians.
• Figure 2: Histograms for the test cases where the fusion approach outperformed the plain sklearn method for (a) test set A and (b) test set B respectively as a measure of the percentage improvement in relative error $\Delta\delta_{V}$ (RS refers to the relaxation schedule of Dirac-3). Note that the axes in the sub-figures have different scales, to appropriately display the range of results for each test set.
• Figure 3: (a) Median relative errors $\delta_{V}$ and (b) number of variables needed per solver/formulation type in Experiment III, shown as functions of matrix size. In (a), the $4\times4$ data point is missing for QUBO because Dirac-3 was not able to accommodate the problem size. Curves are fitted for the data in (b) but not in (a) .
• Figure 4: Histograms for the test cases where Dirac-3 (with QuarDP formulation) outperformed the CP-SAT solver for (a) test set A and (b) test set B respectively as a measure of the percentage improvement in relative error $\varDelta\delta_{V}$.
• Figure 5: Block diagram of the Dirac-3 entropy computer. Here, variable optical attenuators (VOA) help in generating the optical signal along with a pump laser and a mean photon number controller. This feeds into the electro optical modulator (EOM), which creates the temporal wavefunction and implements the loss mechanism with the help of a digital-to-analog converter (DAC). A periodically-poled lithium niobate (PPLN) waveguide does the frequency up-conversion of the photons in the optical signal for better detection. Finally, measurements are taken with the help of a single photon detector and a time-correlated photon counter (for multiple shots) and a candidate solution is created from the tomography. A field programmable gate array (FPGA) then calculates the new loss rate based on the candidate solution measured in the context of the problem Hamiltonian, and the cycle repeats until a termination criteria, such as a timeout or convergence to a candidate solution, is satisfied.