Recursive Sketched Interpolation: Efficient Hadamard Products of Tensor Trains

TL;DR

RSI addresses a bottleneck in TT computations by reducing the Hadamard product cost from $O(χ^4)$ to $O(χ^3)$ through randomized TT sketching and interpolative decomposition. It maintains the TT bond dimension while constructing the product core-by-core in a left-nested, recursive fashion, reusing sketches to avoid large intermediate dimensions. The method generalizes to Hadamard products of multiple TTs and to other element-wise nonlinear mappings, with empirical results across quantum MPSs and quantics TT functions showing favorable scalability and competitive accuracy. This approach enables efficient TT-based nonlinear operations in applications such as nonlinear PDEs, quantum simulations, and TT-represented function multiplication, offering practical speedups for high-rank TT problems.

Abstract

The Hadamard product of two tensors in the tensor-train (TT) format is a fundamental operation across various applications, such as TT-based function multiplication for nonlinear differential equations or convolutions. However, conventional methods for computing this product typically scale as at least $\mathcal{O}(χ^4)$ with respect to the TT bond dimension (TT-rank) $χ$, creating a severe computational bottleneck in practice. By combining randomized tensor-train sketching with slice selection via interpolative decomposition, we introduce Recursive Sketched Interpolation (RSI), a ``scale product'' algorithm that computes the Hadamard product of TTs at a computational cost of $\mathcal{O}(χ^3)$. Benchmarks across various TT scenarios demonstrate that RSI offers superior scalability compared to traditional methods while maintaining comparable accuracy. We generalize RSI to compute more complex operations, including Hadamard products of multiple TTs and other element-wise nonlinear mappings, without increasing the complexity beyond $\mathcal{O}(χ^3)$.

Figures (15)

• Figure 1: The recursive sketched interpolation method offers a $\chi^3$ runtime complexity versus the direct method which scales at least as $\chi^4$. The tensor diagrams illustrate snapshots of the respective computational procedures.
• Figure 2: Tensor diagram notation.
• Figure 3: Cross and interpolative decomposition of a matrix $M$. The cross format can reduce to a one-side ID form. Note that ID factorizations can be computed directly without passing through the cross form.
• Figure 4: Workflow overview of the first iteration of RSI.
• Figure 5: (a) Bond dimensions $\chi_i$, $i=1,\ldots,19$, of the MPS $\psi$ with $\chi_{\max}=10, 20,$ and $30$. (b) Average runtime (ms) versus $\chi_{\max}(\psi)$ for RSI, a single sweep of TCI, and the direct method. (c) Relative error $\epsilon_r$ of the product computed by RSI, TCI, and the direct method as a function of the result's maximum bond dimension $\chi_{\max}(|\psi|^2)$.