Thermodynamic Algorithms for Quadratic Programming

TL;DR

This work introduces a novel approach to solving quadratic programming problems using thermodynamic hardware by incorporating a thermodynamic subroutine for solving linear systems into the interior-point method, and presents a hybrid digital-analog algorithm that outperforms traditional digital algorithms in terms of speed.

Abstract

Thermodynamic computing has emerged as a promising paradigm for accelerating computation by harnessing the thermalization properties of physical systems. This work introduces a novel approach to solving quadratic programming problems using thermodynamic hardware. By incorporating a thermodynamic subroutine for solving linear systems into the interior-point method, we present a hybrid digital-analog algorithm that outperforms traditional digital algorithms in terms of speed. Notably, we achieve a polynomial asymptotic speedup compared to conventional digital approaches. Additionally, we simulate the algorithm for a support vector machine and predict substantial practical speedups with only minimal degradation in solution quality. Finally, we detail how our method can be applied to portfolio optimization and the simulation of nonlinear resistive networks.

Figures (4)

• Figure 1: Overview of the hybrid digital-analog algorithm to solve quadratic programs. At initialization, $\tilde{J}_1 = J_1^\top J_1$ and $v_1$ are computed by a GPU (to benefit from the high parallelization of matrix-matrix and matrix-vector multiplications) and uploaded onto the SPU. After some dynamical evolution, the update $\Delta \bm{r}_1$ is downloaded from the SPU, which enables the calculation of quantities $X_2, Z_2, X_2Z_2$ which are used to upload the matrix representation of $\tilde{J}$ (only updating blocks of it), and $v_2$ which are then downloaded onto the SPU. This continues until a satisfactory solution is reached, according to the criteria described in \ref{['alg:2']}.
• Figure 2: Comparison of digital and thermodynamic approaches to training SVMs. The left panel compares the training times when different subroutines for the IPM are used. The thermodynamic subroutine runtime is estimated with a timing model, detailed in Appendix \ref{['app:hardware']}. The right panel shows the associated training accuracy of the model, showing a minimal degradation in solution quality. Here regularization is added for both the CG and thermodynamic IPMs, to stabilize the linear system $\tilde{J}_k \bm{\Delta r}_k = \bm{\tilde{v}}_k$, with parameter $\lambda = 0.1$. The simulations were performed on an Nvidia A100 GPU.
• Figure 3: Quadratic programming approach to resisitve networks. The parameters of the resistive networks (i.e. voltage sources $V$, current sources $I$, diodes $D$, resistors $R$ and the graph structure $G$ of the network) are encoded in the parameters of a quadratic program ($A$, $\bm{b}$ and $\bm{c}$). The thermodynamic interior point algorithm (Algorithm \ref{['alg:2']}) is then used to solve the optimization problem and produces a close-to-optimal solution $\bm{x}^{\star}$. The solution encodes the information about the steady state configuration (node voltages) of the resistive network.
• Figure 4: Circuit diagram for the thermodynamic device in the case of a single resistor array implementation for a 3-dimensional problem. The device is comprised of three voltage sources, each of which is connected to three resistors $\{R_i\}$ . Each of these resistors is connected to a line that goes into the negative pin of a different operational amplifier. A capacitor connects the negative pin of the operational amplifier to the operational amplifier's output port, where the voltage is denoted by $\{V_i\}$. The output ports of the operational amplifiers are connected to an array of $N\times N$ resistors $\{R_{ij}\}$ (nine here), each of which connects to the line going from the resistors $\{R_i\}$ to the negative pins of the operational amplifiers. This feedback loop enables the circuit to run a differential equation whose steady-state corresponds to the solution of a linear system of equations.