Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Randomized Algorithm for Tensor Singular Value Decomposition using an Arbitrary Number of Passes

Salman Ahmadi-Asl, Anh-Huy Phan, Andrzej Cichocki

TL;DR

The paper tackles the cost of computing tensor SVD (t-SVD) for large-scale data by enabling an arbitrary budget of data passes, addressing the limitation that traditional randomized methods require $2q+2$ passes. It extends pass-efficient randomized matrix techniques to the tensor setting via the t-product, introducing a pass-efficient truncated t-SVD that works for any budget $v$, including odd values, and provides average Frobenius error bounds that depend on the spectral gap $\tau_R = \sigma_{R+1}/\sigma_R$. The authors derive theoretical guarantees and demonstrate practical performance through simulations on synthetic and real data, showing notable speedups over baseline methods while maintaining competitive accuracy, and they apply the approach to tensor completion tasks. This work enables scalable, flexible tensor decompositions for large-scale data and repeated analyses in applications such as image/video completion and reconstruction.

Abstract

Efficient and fast computation of a tensor singular value decomposition (t-SVD) with a few passes over the underlying data tensor is crucial because of its many potential applications. The current/existing subspace randomized algorithms need (2q+2) passes over the data tensor to compute a t-SVD, where q is a non-negative integer number (power iteration parameter). In this paper, we propose an efficient and flexible randomized algorithm that can handle any number of passes q, which not necessary need be even. The flexibility of the proposed algorithm in using fewer passes naturally leads to lower computational and communication costs. This advantage makes it particularly appropriate when our task calls for several tensor decompositions or when the data tensors are huge. The proposed algorithm is a generalization of the methods developed for matrices to tensors. The expected/ average error bound of the proposed algorithm is derived. Extensive numerical experiments on random and real-world data sets are conducted, and the proposed algorithm is compared with some baseline algorithms. The extensive computer simulation experiments demonstrate that the proposed algorithm is practical, efficient, and in general outperforms the state of the arts algorithms. We also demonstrate how to use the proposed method to develop a fast algorithm for the tensor completion problem.

A Randomized Algorithm for Tensor Singular Value Decomposition using an Arbitrary Number of Passes

TL;DR

The paper tackles the cost of computing tensor SVD (t-SVD) for large-scale data by enabling an arbitrary budget of data passes, addressing the limitation that traditional randomized methods require passes. It extends pass-efficient randomized matrix techniques to the tensor setting via the t-product, introducing a pass-efficient truncated t-SVD that works for any budget , including odd values, and provides average Frobenius error bounds that depend on the spectral gap . The authors derive theoretical guarantees and demonstrate practical performance through simulations on synthetic and real data, showing notable speedups over baseline methods while maintaining competitive accuracy, and they apply the approach to tensor completion tasks. This work enables scalable, flexible tensor decompositions for large-scale data and repeated analyses in applications such as image/video completion and reconstruction.

Abstract

Efficient and fast computation of a tensor singular value decomposition (t-SVD) with a few passes over the underlying data tensor is crucial because of its many potential applications. The current/existing subspace randomized algorithms need (2q+2) passes over the data tensor to compute a t-SVD, where q is a non-negative integer number (power iteration parameter). In this paper, we propose an efficient and flexible randomized algorithm that can handle any number of passes q, which not necessary need be even. The flexibility of the proposed algorithm in using fewer passes naturally leads to lower computational and communication costs. This advantage makes it particularly appropriate when our task calls for several tensor decompositions or when the data tensors are huge. The proposed algorithm is a generalization of the methods developed for matrices to tensors. The expected/ average error bound of the proposed algorithm is derived. Extensive numerical experiments on random and real-world data sets are conducted, and the proposed algorithm is compared with some baseline algorithms. The extensive computer simulation experiments demonstrate that the proposed algorithm is practical, efficient, and in general outperforms the state of the arts algorithms. We also demonstrate how to use the proposed method to develop a fast algorithm for the tensor completion problem.
Paper Structure (11 sections, 8 theorems, 34 equations, 13 figures, 4 tables, 6 algorithms)

This paper contains 11 sections, 8 theorems, 34 equations, 13 figures, 4 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Tensor SVD and Tensor QR decomposition
  4. A Pass-Efficient randomized algorithm for computation of the t-SVD
  5. An Application to tensor completion
  6. Simulations
  7. Conclusion and future works
  8. Acknowledgement
  9. Conflict of Interest Statement
  10. Data Availability
  11. Appendix

Key Result

Theorem 1

zhang2018randomized (Average Frobenius error for Algorithm ALg_1). Let ${\bf X}\in\mathbb{R}^{I_1\times I_2}$ and ${\bf \Omega}\in\mathbb{R}^{I_2\times (R+P)}$ be a given matrix and a Gaussian random matrix, respectively with the oversampling parameter $P \geq 2$. Suppose ${\bf Q}$ is obtained from where $R$ is a matrix rank, $q$ is the power iteration, $\sigma_j$ is the $j$-th singular value of

Figures (13)

  • Figure 1: Illustration of ( a) Tensor SVD (t-SVD) and ( b) truncated t-SVD for a third-order tensor ahmadi2021cross.
  • Figure 2: Running time and relative error of the approximations achieved by the proposed algorithm for a synthetic data tensor of size $500\times 500\times 500$ and the tubal rank $R=15$ using different numbers of passes for Example \ref{['Ex_1']}.
  • Figure 3: Running time and relative error of the approximations achieved by the truncated t-SVD, the randomized t-SVD and the proposed algorithm for a synthetic data tensor of size $500\times 500\times 500$ for different tubal ranks for Example \ref{['Ex_1']}.
  • Figure 4: (Upper) The reconstruction of the "Kodim23" and the "Kodim03" images using the proposed algorithm for the tubal rank $R=20$ and different numbers of passes (Bottom) The running time of the proposed algorithm for computation of the truncated t-SVD of the "Kodim23" image (left) and the "Kodim03" image (right) with the tubal rank $R=40$ and using different numbers of passes for Example \ref{['salman']}. .
  • Figure 5: The running time and PSNRs of the reconstructed images achieved by the truncated t-SVD, the randomized t-SVD and the proposed algorithm using different tubal ranks for Example \ref{['salman']}.
  • ...and 8 more figures

Theorems & Definitions (21)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Theorem 1
  • Theorem 2
  • proof
  • Theorem 3
  • ...and 11 more