Parametric Matrix Models

TL;DR

It is proved that parametric matrix models are universal function approximators that can be applied to general machine learning problems and excel at making accurate predictions for scientific computing applications.

Abstract

We present a general class of machine learning algorithms called parametric matrix models. In contrast with most existing machine learning models that imitate the biology of neurons, parametric matrix models use matrix equations that emulate physical systems. Similar to how physics problems are usually solved, parametric matrix models learn the governing equations that lead to the desired outputs. Parametric matrix models can be efficiently trained from empirical data, and the equations may use algebraic, differential, or integral relations. While originally designed for scientific computing, we prove that parametric matrix models are universal function approximators that can be applied to general machine learning problems. After introducing the underlying theory, we apply parametric matrix models to a series of different challenges that show their performance for a wide range of problems. For all the challenges tested here, parametric matrix models produce accurate results within an efficient and interpretable computational framework that allows for input feature extrapolation.

Figures (8)

• Figure 1: Left Panel: Performance on regression problems. Normalized mean absolute error for the PMM (blue) compared against several standard techniques: Kernel Ridge Regression (KRR, orange), Multilayer Perceptron (MLP, green), $k$-Nearest Neighbors (KNN, red), Extreme Gradient Boosting (XGB, purple), Support Vector Regression (SVR, brown), and Random Forest Regression (RFR, pink). Mean performance across all problems are shown as dashed lines. Right Panel: Extrapolated Trotter approximation for quantum computing simulations. We plot the lowest three energies of the effective Hamiltonian for the one-dimensional Heisenberg model with DM interactions versus time step $dt$. We compare results obtained using a PMM, Multilayer Perceptron (MLP), and polynomial interpolation (Poly). All training (diamonds) and validation (circles) samples are located away from $dt=0$, where data acquisition on a quantum computer would be practical. The inset shows the relative error in the predicted energies at $dt=0$ for the three models.
• Figure 2: Left Panel: Lowest two energy levels of the ALMG model versus $\xi$. We show PMM results compared with Multilayer Perceptron (MLP) results. The upper plots show the energies, and the lower plots show absolute error. The main plots show the region around the phase transition; the insets show the full domain where data was provided. Right Panel: Average particle density for the ground state of the ALMG model versus $\xi$. We show PMM results compared with Multilayer Perceptron (MLP) results. The upper plots show the average particle density, and the lower plots show absolute error. The main plots show the region around the phase transition; the insets show the full domain where data was provided.
• Figure 3: Complex-valued ground state energy of the ALMG model for complex $\xi$. We show PMM predictions for the complex-valued ground state energy for complex values of $\xi$, using training data at only real values of $\xi$. The left plot shows the exact results, the middle plot shows the PMM predictions, and the right plot shows the absolute error.
• Figure S1: Concatenated line segments for one input feature. The thick line shows a particular eigenvalue $\lambda(c_1)$ that traces out a function composed of several concatenated line segments. The dashed lines show the affine functions $f_j(c_1) = a_j c_1 + b_j$ that describe the line segments.
• Figure S2: Comparison of PMM and EC results for ground state energy extrapolation. We show results for a $2\times 2$ PMM (dashed blue) and EC (dotted red) with $5$ training samples on the task of extrapolating the ground state energy of a system of $N$ non-interacting spins. The exact ground state energy is shown in solid black.