Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized algorithms for Kroncecker tensor decomposition and applications

Salman Ahmadi-Asl, Naeim Rezaeian, Andre L. F. de Almeida, Yipeng Liu

TL;DR

This paper tackles the scalability challenge of computing the Kronecker Tensor Decomposition (KTD) for large-scale tensors by introducing fast randomized algorithms that replace costly SVD steps with randomized CPD approaches. The authors develop a randomized KTD framework (rKTD) with oversampling and power iteration to achieve substantial computational speedups, including variants that use Tucker compression to further accelerate processing. They provide theoretical error bounds and demonstrate, through extensive simulations on synthetic and real data (image/video completion, denoising, and super-resolution), that the randomized methods achieve orders-of-magnitude faster runtimes with competitive accuracy. The work highlights practical impact for large-scale tensor analysis and points to future extensions in recommender systems and neural network model compression, along with potential CUR-based speedups. Overall, the proposed randomized KTD offers a principled, efficient pathway to harnessing KTD in real-world, high-dimensional data settings.

Abstract

This paper proposes fast randomized algorithms for computing the Kronecker Tensor Decomposition (KTD). The proposed algorithms can decompose a given tensor into the KTD format much faster than the existing state-of-the-art algorithms. Our principal idea is to use the randomization framework to reduce computational complexity significantly. We provide extensive simulations to verify the effectiveness and performance of the proposed randomized algorithms with several orders of magnitude acceleration compared to the deterministic one. Our simulations use synthetics and real-world datasets with applications to tensor completion, video/image compression, image denoising, and image super-resolution

Randomized algorithms for Kroncecker tensor decomposition and applications

TL;DR

This paper tackles the scalability challenge of computing the Kronecker Tensor Decomposition (KTD) for large-scale tensors by introducing fast randomized algorithms that replace costly SVD steps with randomized CPD approaches. The authors develop a randomized KTD framework (rKTD) with oversampling and power iteration to achieve substantial computational speedups, including variants that use Tucker compression to further accelerate processing. They provide theoretical error bounds and demonstrate, through extensive simulations on synthetic and real data (image/video completion, denoising, and super-resolution), that the randomized methods achieve orders-of-magnitude faster runtimes with competitive accuracy. The work highlights practical impact for large-scale tensor analysis and points to future extensions in recommender systems and neural network model compression, along with potential CUR-based speedups. Overall, the proposed randomized KTD offers a principled, efficient pathway to harnessing KTD in real-world, high-dimensional data settings.

Abstract

This paper proposes fast randomized algorithms for computing the Kronecker Tensor Decomposition (KTD). The proposed algorithms can decompose a given tensor into the KTD format much faster than the existing state-of-the-art algorithms. Our principal idea is to use the randomization framework to reduce computational complexity significantly. We provide extensive simulations to verify the effectiveness and performance of the proposed randomized algorithms with several orders of magnitude acceleration compared to the deterministic one. Our simulations use synthetics and real-world datasets with applications to tensor completion, video/image compression, image denoising, and image super-resolution

Paper Structure

This paper contains 10 sections, 4 theorems, 23 equations, 13 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Kronecker tensor decomposition
  4. Randomized algorithms for low-rank matrix approximation
  5. Proposed fast randomized Kronecker tensor decomposition
  6. Computational complexity
  7. Simulations
  8. Conclusion and future works
  9. Conflict of Interest Statement
  10. Data availability Statement

Key Result

Lemma 1

Let ${\bf a},\,{\bf b}$ and ${\bf c}$ be three vectors; then ${\rm vec}({\bf a}\circ{\bf b}\circ{\bf c})={\bf c}\otimes{\bf b}\otimes{\bf a}$.

Figures (13)

  • Figure 1: Fast CPD with a fast prior Tucker compression.
  • Figure 2: (Left) Running time comparison of the proposed randomized algorithms and the deterministic algorithm for decomposing a fourth-order tensor of size $100\times 100\times 100\times 100$ for different KTD rank $R=10,20,30,40,50$. (Right) Running time comparison of the proposed randomized algorithms and the deterministic algorithm for decomposing a 3rd order tensor of size $1000\times 1000\times 1000$ for different KTD rank $R=10,20,30,40,50.$
  • Figure 3: Comparing the image quality obtained by the deterministic KTD and the proposed R-KTD using different power iterations $q=0,1,2$. The KTD rank $R=35$ was used.
  • Figure 4: (left) Running time comparison for compressing the kodim23 using the deterministic KTD and the proposed R-KTD for different KTD ranks and the power iteration $q=1$. (right) The PSNRs of all compressed Kodak images for the deterministic KTD and the proposed R-KTD methods.
  • Figure 5: Comparing the PSNRs achieved by the deterministic KTD and the proposed R-KTD for compressing the Foreman (left) and the Aikyo (right) videos. The KTD rank $R=40$ was used.
  • ...and 8 more figures

Theorems & Definitions (12)

  • Definition 1
  • Definition 2
  • Lemma 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • proof
  • Example 1
  • Example 2
  • Example 3
  • ...and 2 more