Linear-Scaling Tensor Train Sketching

TL;DR

BSTT is introduced, a structured random projection tailored to the tensor train (TT) format that unifies existing TT-adapted sketching operators and derives quasi-optimal error bounds for the QB factorization and randomized TT rounding.

Abstract

We introduce the Block Sparse Tensor Train (BSTT) sketch, a structured random projection tailored to the tensor train (TT) format that unifies existing TT-adapted sketching operators. By varying two integer parameters $P$ and $R$, BSTT interpolates between the Khatri-Rao sketch ($R=1$) and the Gaussian TT sketch ($P=1$). We prove that BSTT satisfies an oblivious subspace embedding (OSE) property with parameters $R = \mathcal{O}(d(r+\log 1/δ))$ and $P = \mathcal{O}(\varepsilon^{-2})$, and an oblivious subspace injection (OSI) property under the condition $R = \mathcal{O}(d)$ and $P = \mathcal{O}(\varepsilon^{-2}(r + \log r/δ))$. Both guarantees depend only linearly on the tensor order $d$ and on the subspace dimension $r$, in contrast to prior constructions that suffer from exponential scaling in $d$. As direct consequences, we derive quasi-optimal error bounds for the QB factorization and randomized TT rounding. The theoretical results are supported by numerical experiments on synthetic tensors, Hadamard products, and a quantum chemistry application.

Figures (8)

• Figure 1: BSTT Framework on Rank-1 Basis. Empirical injectivity (left) and dilation (right) of the proposed BSTT sketches. The operators are evaluated using embedding dimensions $PR = 2r$ and block ranks $R\in\{1,4,16,32\}$, applied to $r$-dimensional target subspaces spanned by Kronecker (rank-1) Gaussian TT vectors in $(\mathbb{R}^4)^{\otimes d}$. Markers indicate the median across 100 independent trials, with shaded regions denoting the interquartile range (25th to 75th percentiles).
• Figure 2: BSTT Framework on Rank-4 Basis. Empirical injectivity (left) and dilation (right) of the proposed BSTT sketches. The operators are evaluated using embedding dimensions $PR = 2r$ and block ranks $R\in\{1,4,16,32\}$, applied to $r$-dimensional target subspaces spanned by rank-$4$ Gaussian TT vectors in $\mathbb{R}^{4^d}$. Markers indicate the median across 100 independent trials, with shaded regions denoting the interquartile range (25th to 75th percentiles).
• Figure 3: Numerical OSE of BSTT Framework on Rank-1 Basis Empirical injectivity (left) and dilation (right) of the proposed BSTT sketches. The operators are evaluated using embedding dimensions $PR$ with $P \in \{1,4,16\}$ blocks of rank $R = 2d$, applied to a $16$-dimensional target subspace spanned by Kronecker (rank-1) Gaussian TT vectors in $(\mathbb{R}^4)^{\otimes d}$. Markers indicate the median across 100 independent trials, with shaded regions denoting the interquartile range (25th to 75th percentiles).
• Figure 4: Graphical representation of the sketching of the matrix-vector product $\mathbf{H}\mathbf{y}$. The matrix $\mathbf{H}$ has a TT representation $(\mathcal{H}_k)_{k \in [d]}$ of ranks $r_H$, $\mathbf{y}$ has a TT representation $(\mathcal{Y}_k)_{k \in [d]}$ of ranks $\chi$, ${\boldsymbol \Omega}_\mathtt{BSTT} \in \mathbb{F}^{PR \times N}$. The contractions are done from right to left, top to bottom.
• Figure 5: Graphical representation of the partial contraction between a Hadamard product $\mathcal{Y}_1 \bullet \mathcal{Y}_2$ and a TT sketch. The figure on the right highlights the dashed section on the left, illustrating the benefit of not assembling the intermediate TT core $\mathcal{C}_{k,1} \bullet \mathcal{C}_{k,2}$.

Theorems & Definitions (74)

• Definition 2.1: Tensor Train Format
• Definition 2.2: Tensor Unfolding
• Definition 2.3: Strong Kronecker Product
• Definition 2.4: Oblivious Subspace Embedding
• Definition 2.5
• Definition 2.6: Oblivious Subspace Injection
• Definition 3.1: Block-Sparse Tensor Train Sketch
• Remark 3.2
• Remark 3.3
• Definition 3.4: Orthogonal Block-Sparse Tensor Train Sketch