Randomized algorithms for streaming low-rank approximation in tree tensor network format

TL;DR

The paper develops TTNN, a streamable, randomized Nyström-based method for low-rank tensor approximation in tree tensor networks, unifying and extending GN approaches to TTN formats and encompassing MLN and STTA as special cases. It provides a rigorous deterministic error bound for TTNN and a probabilistic bound under Gaussian dimension-reduction maps, along with a deterministic bound for the sequential STTNN variant, which excels for dense tensors. Structured sketchings via Khatri-Rao embeddings are proposed to exploit TTN structure and reduce computational cost, and STTNN is shown to offer substantial runtime gains while preserving accuracy close to TTNN. Numerical experiments on dense and synthetic TTN tensors demonstrate competitive accuracy with TTN-SVD and clear speedups, underscoring the practical impact for streaming, scalable tensor compression and rounding. The work opens avenues for randomized solvers in general TTN-like networks and potential extensions to cyclic tensor networks such as tensor rings and MERA.

Abstract

In this work, we present the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation, such as for Tucker and tensor-train formats, to the more general tree tensor network format, enabling a unified treatment of various existing methods. Our method retains the key features of the generalized Nyström approximation for matrices, that is randomized, single-pass, streamable, and cost-effective. Additionally, the structure of the sketching allows for parallel implementation. We provide a deterministic error bound for the algorithm and, in the specific case of Gaussian dimension reduction maps, also a probabilistic one. We also introduce a sequential variant of the algorithm, referred to as sequential tree tensor network Nyström (STTNN), which offers better performance for dense tensors. Furthermore, both algorithms are well-suited for the recompression or rounding of tensors in the tree tensor network format. Numerical experiments highlight the efficiency and effectiveness of the proposed methods.

Figures (10)

• Figure 1: An example of an index tree.
• Figure 1: TTNN approximation of a six-mode tensor with the index tree in figure \ref{['fig: example of index tree']}.
• Figure 1: Tree diagrams illustrating the ordering of nodes for which projections are computed (circled) and the indices of previously computed contractions that can be exploited (boxed).
• Figure 1: Graphical illustration of how to use Khatri-Rao embeddings to compute the entries of $\Omega_{1,1}$ (left) and $\Psi_{1,1}$ (right) exploiting the tree structure.
• Figure 1: Frobenius error of approximation (left) and running time (right) obtained by the TTNN algorithm with the index tree in Figure \ref{['fig: example of index tree']} and Gaussian sketchings for different values of $r$ on the 6D Hilbert tensor of size $20\times 20 \times 20 \times 20 \times 20 \times 20$.

Theorems & Definitions (18)

• Definition 3.1: mode-$k$ product
• Definition 3.2: Matricization
• Definition 3.3: index tree
• Definition 3.4
• Definition 3.5: Tree tensor network format
• Lemma 6.1
• Proof 1
• Lemma 6.2
• Proof 2
• Lemma 6.3