Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized algorithms for computing the tensor train approximation and their applications

Maolin Che, Yimin Wei, Hong Yan

TL;DR

This work tackles computing tensor train (TT) approximations under two settings: fixed TT-rank and fixed precision. It introduces two randomized TT frameworks based on random projections and power iterations, with theoretical error bounds that apply to standard Gaussian and Khatri-Rao Gaussian matrices. The paper also presents an adaptive epsilon-TT-rank estimator and an efficient adaptive randomized TT method, plus a greedy TT-SVD variant for fixed-precision scenarios. Empirical results on synthetic and real tensors show substantial speedups over TT-SVD while maintaining competitive accuracy, and demonstrate robustness across multiple random matrix choices and data types.

Abstract

In this paper, we focus on the fixed TT-rank and precision problems of finding an approximation of the tensor train (TT) decomposition of a tensor. Note that the TT-SVD and TT-cross are two well-known algorithms for these two problems. Firstly, by combining the random projection technique with the power scheme, we obtain two types of randomized algorithms for the fixed TT-rank problem. Secondly, by using the non-asymptotic theory of sub-random Gaussian matrices, we derive the upper bounds of the proposed randomized algorithms. Thirdly, we deduce a new deterministic strategy to estimate the desired TT-rank with a given tolerance and another adaptive randomized algorithm that finds a low TT-rank representation satisfying a given tolerance, and is beneficial when the target TT-rank is not known in advance. We finally illustrate the accuracy of the proposed algorithms via some test tensors from synthetic and real databases. In particular, for the fixed TT-rank problem, the proposed algorithms can be several times faster than the TT-SVD, and the accuracy of the proposed algorithms and the TT-SVD are comparable for several test tensors.

Randomized algorithms for computing the tensor train approximation and their applications

TL;DR

This work tackles computing tensor train (TT) approximations under two settings: fixed TT-rank and fixed precision. It introduces two randomized TT frameworks based on random projections and power iterations, with theoretical error bounds that apply to standard Gaussian and Khatri-Rao Gaussian matrices. The paper also presents an adaptive epsilon-TT-rank estimator and an efficient adaptive randomized TT method, plus a greedy TT-SVD variant for fixed-precision scenarios. Empirical results on synthetic and real tensors show substantial speedups over TT-SVD while maintaining competitive accuracy, and demonstrate robustness across multiple random matrix choices and data types.

Abstract

In this paper, we focus on the fixed TT-rank and precision problems of finding an approximation of the tensor train (TT) decomposition of a tensor. Note that the TT-SVD and TT-cross are two well-known algorithms for these two problems. Firstly, by combining the random projection technique with the power scheme, we obtain two types of randomized algorithms for the fixed TT-rank problem. Secondly, by using the non-asymptotic theory of sub-random Gaussian matrices, we derive the upper bounds of the proposed randomized algorithms. Thirdly, we deduce a new deterministic strategy to estimate the desired TT-rank with a given tolerance and another adaptive randomized algorithm that finds a low TT-rank representation satisfying a given tolerance, and is beneficial when the target TT-rank is not known in advance. We finally illustrate the accuracy of the proposed algorithms via some test tensors from synthetic and real databases. In particular, for the fixed TT-rank problem, the proposed algorithms can be several times faster than the TT-SVD, and the accuracy of the proposed algorithms and the TT-SVD are comparable for several test tensors.
Paper Structure (23 sections, 16 theorems, 81 equations, 9 figures, 4 tables, 5 algorithms)

This paper contains 23 sections, 16 theorems, 81 equations, 9 figures, 4 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Notations and organizations
  3. Preliminaries
  4. TT decomposition
  5. Randomized algorithms for the fixed-TT-rank problem
  6. Framework for the first algorithm
  7. Framework of the second algorithm
  8. Theoretical analysis
  9. The case for Algorithm \ref{['RTT:alg3']} with standard Gaussian matrices
  10. The case for Algorithm \ref{['RTT:alg3']} with the Khatri-Rao product of standard Gaussian matrices
  11. The case for Algorithm \ref{['RTT:alg2']}
  12. Comparison of the upper bounds of TT-SVD and the proposed algorithms
  13. Randomized algorithms for the fixed precision problem
  14. Greedy TT-SVD
  15. An efficient adaptive randomized algorithm
  16. ...and 8 more sections

Key Result

Theorem 4.1

(see litvak2005smallest) Suppose that $\mathbf{\Omega}\in \mathbb{R}^{I\times J}$ is random sub-Gaussian with $I\leq J$, $\tau\geq1$ and $a'>0$. Then $\mathbf{P}(\|\mathbf{\Omega}\|_2>a\sqrt{J})\leq \exp(-a'J)$, where $a=6\tau\sqrt{a'+4}$.

Figures (9)

  • Figure 1: Illustration for the way to find an approximate solution for Problem \ref{['RTT:prob2']} with $N=4$. In the diagram, RE, ON and UP means reshaping, an orthonormal basis, and updating, respectively.
  • Figure 2: For different oversampling parameters $R$, the results by applying Algorithm \ref{['RTT:alg3']} with Gaussian (top row), Algorithm \ref{['RTT:alg3']} with KR-Gaussian (middle row) and Algorithm \ref{['RTT:alg2']} (bottom row) to $\mathcal{A}$ with $\mu=2,4,6,\dots,20$.
  • Figure 3: For different $q$, the results by applying Algorithm \ref{['RTT:alg3']} with Gaussian (top row), Algorithm \ref{['RTT:alg3']} with KR-Gaussian (middle row) and Algorithm \ref{['RTT:alg2']} (bottom row) to $\mathcal{A}$ with $\mu=2,4,6,\dots,20$.
  • Figure 4: The results by applying Algorithm \ref{['RTT:alg3']} with Gaussian, KR-Gaussian, Kron-Gaussian, SpEmb and SDCT, and Algorithm \ref{['RTT:alg2']} to $\mathcal{A}$ (top row) with $\mu=2,4,6,\dots,20$, and $\mathcal{B}$ (bottom row) with ${\rm SNR}= -30,-25,\dots,25,30$.
  • Figure 5: The results by applying Algorithm \ref{['RTT:alg3']} with Gaussian, KR-Gaussian, and Algorithm \ref{['RTT:alg2']} to $\mathcal{A}$ with $\mu=2,4,6,\dots,20$. Error bars show one standard deviation.
  • ...and 4 more figures

Theorems & Definitions (38)

  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Definition 4.1
  • Theorem 4.1
  • Theorem 4.2
  • Lemma 4.1
  • proof
  • ...and 28 more