Fast elementwise operations on tensor trains with alternating cross interpolation

Abstract

Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as $O(χ^4)$, where $χ$ is the TT rank. If the rank of the output is smaller than $χ^2$, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in $O(χ^3)$, while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.

Figures (3)

• Figure 1: Multiplication of Gaussians for algorithm verification. (a) The input functions $g_\pm$ (dashed lines), given by Eq. \ref{['eq:gaussians']}, and their product $h$, computed using ACI (solid lines). We set $w = 0.15$ and vary $\delta \in \{0.1, 0.4, 0.8\}$, discretize with $\mathcal{L} = 25$ binary indices, and perform ACI with $\chi' = 15$ on TTs generated with 2-site TCI. (b) Maximum error between the exact product $g_+ \odot g_-$ and the ACI output $h$, as a function of output bond dimension $\chi'$.
• Figure 2: Hadamard product of random Fourier series, Eq. \ref{['eq:fouriermultiplication']}. Two functions of the form \ref{['eq:fourierseries']} are represented as TT with $\mathcal{L} = 30$ quantics indices, then multiplied elementwise with a tolerance of $\tau = 10^{-8}$ using the ACI algorithm (this work, blue circles), and an algorithm based on MPO-MPS contraction (yellow triangles). (a) Runtime needed to perform the multiplication. Lines are fitted power laws $\sim\chi^p$, which are consistent with the theoretical scaling of $\mathcal{O}(\chi^3)$ for ACI and $\mathcal{O}(\chi^4)$ for the contraction-based Hadamard product. Runtimes were measured using a single thread on an AMD EPYC 9474F processor. (b) Maximum error between the ACI output $h$ and the exact product $f\odot g$. Both methods consistently reach the error tolerance $\tau = 10^{-8}$ (black line).
• Figure 3: Similar to Fig. \ref{['fig:fourier']}(a), but now for the Hadamard product of random TT. Instead of setting a fixed tolerance, the output bond dimension was set to $\chi' = \chi$ here.