Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Algorithms for Low Tubal Rank Tensor Approximation with Applications

Salman Ahmadi-Asl, Naeim Rezaeian, Cesar F. Caiafa, Andre L. F. de Almeidad

TL;DR

The paper tackles scalable low tubal rank tensor approximation under the $T$-product by introducing both randomized single-pass and fixed-precision algorithms for the tensor SVD (T-SVD). It contributes three robust single-pass methods that stabilize range/co-range sketches via truncated $T$-SVD and two faster fixed-precision schemes that reduce data passes and leverage $T$-EIG and tensor inverses. Theoretical guarantees accompany the algorithms, and extensive experiments on synthetic data, image/video compression, super-resolution, and deep-learning preprocessing demonstrate improved speed and robustness over baselines. The work showcases practical impact for large-scale tensor tasks where memory and pass efficiency are critical. Mathematics are presented in the $T$-product framework, with tubal rank guiding truncation and approximation accuracy.

Abstract

In this paper we propose efficient randomized fixed-precision techniques for low tubal rank approximation of tensors. The proposed methods are faster and more efficient than the existing fixed-precision algorithms for approximating the truncated tensor SVD (T-SVD). Besides, there are a few works on randomized single-pass algorithms for computing low tubal rank approximation of tensors, none of them experimentally reports the robustness of such algorithms for low-rank approximation of real-world data tensors e.g., images and videos. The current single-pass algorithms for tensors are generalizations of those developed for matrices to tensors. However, the single-pass randomized algorithms for matrices have been recently improved and stabilized. Motivated by this progress, in this paper, we also generalize them to the tensor case based on the tubal product (T-product). We conduct extensive simulations to study the robustness of them compared with the existing single-pass randomized algorithms. In particular, we experimentally found that single-pass algorithms with the sketching parameters of equal sizes usually lead to ill-conditioned tensor least-squares problems and inaccurate results. It is experimentally shown that our proposed single-pass algorithms are robust in this sense. Numerical results demonstrate that under the same conditions (setting the same hyper-parameters), our proposed algorithms provide better performance. Three applications to image compression, super-resolution problem and deep learning are also presented.

Efficient Algorithms for Low Tubal Rank Tensor Approximation with Applications

TL;DR

The paper tackles scalable low tubal rank tensor approximation under the -product by introducing both randomized single-pass and fixed-precision algorithms for the tensor SVD (T-SVD). It contributes three robust single-pass methods that stabilize range/co-range sketches via truncated -SVD and two faster fixed-precision schemes that reduce data passes and leverage -EIG and tensor inverses. Theoretical guarantees accompany the algorithms, and extensive experiments on synthetic data, image/video compression, super-resolution, and deep-learning preprocessing demonstrate improved speed and robustness over baselines. The work showcases practical impact for large-scale tensor tasks where memory and pass efficiency are critical. Mathematics are presented in the -product framework, with tubal rank guiding truncation and approximation accuracy.

Abstract

In this paper we propose efficient randomized fixed-precision techniques for low tubal rank approximation of tensors. The proposed methods are faster and more efficient than the existing fixed-precision algorithms for approximating the truncated tensor SVD (T-SVD). Besides, there are a few works on randomized single-pass algorithms for computing low tubal rank approximation of tensors, none of them experimentally reports the robustness of such algorithms for low-rank approximation of real-world data tensors e.g., images and videos. The current single-pass algorithms for tensors are generalizations of those developed for matrices to tensors. However, the single-pass randomized algorithms for matrices have been recently improved and stabilized. Motivated by this progress, in this paper, we also generalize them to the tensor case based on the tubal product (T-product). We conduct extensive simulations to study the robustness of them compared with the existing single-pass randomized algorithms. In particular, we experimentally found that single-pass algorithms with the sketching parameters of equal sizes usually lead to ill-conditioned tensor least-squares problems and inaccurate results. It is experimentally shown that our proposed single-pass algorithms are robust in this sense. Numerical results demonstrate that under the same conditions (setting the same hyper-parameters), our proposed algorithms provide better performance. Three applications to image compression, super-resolution problem and deep learning are also presented.

Paper Structure

This paper contains 8 sections, 1 theorem, 30 equations, 10 figures, 4 tables, 11 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Generalization of standard matrix decompositions to tensors through the T-product
  4. Proposed randomized single-pass algorithms
  5. Proposed randomized fixed-precision algorithms
  6. Experimental results
  7. Conclusion
  8. Conflict of Interest Statement

Key Result

Theorem 1

Let $\underline{\bf X}\in\mathbb{R}^{I_1\times I_2\times I_3}$ and $\underline{\bf \Omega}\in\mathbb{R}^{I_2\times k\times I_3}$ be a random tensor, $\underline{\bf Y}=\underline{\bf X}*\underline{\bf \Omega},\,\underline{\bf W}=\underline{\bf X}^T*\underline{\bf Y}$, and the economic T-SVD of $\und gives an approximation $\underline{\bf Q}*\underline{\bf B}$ to $\underline{\bf X}$ that has the sa

Figures (10)

  • Figure 1: Randomized low tubal rank approximation based on slice sampling tarzanagh2018fastahmadi2021cross.
  • Figure 2: Illustration of ( a) Tensor SVD (T-SVD) and ( b) truncated T-SVD for a third-order tensor .
  • Figure 3: Reconstructed images using different single-pass algorithms for sketch sizes $L=350,\,K=350,\,H=100$ and $R=30$. The results show a better performance of the proposed randomized single-pass algorithms.
  • Figure 4: The PSNR of all frames of the Foreman video was achieved using different single-pass algorithms for sketch sizes $L=90, K=90, H=20$ and $R=50$. The results show a better performance of the proposed randomized single-pass algorithms.
  • Figure 5: The PSNR of all frames of the News Video was achieved using different single-pass algorithms for sketch sizes $L=90, K=90, H=20$ and $R=50$. The results show a better performance of the proposed randomized single-pass algorithms.
  • ...and 5 more figures

Theorems & Definitions (14)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Theorem 1
  • proof
  • Example 1
  • ...and 4 more