Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent

Zhiyu Liu, Zhi Han, Yandong Tang, Xi-Le Zhao, Yao Wang

TL;DR

The paper tackles low-tubal-rank tensor recovery from limited, noisy measurements by introducing a Burer-Monteiro–style tensor factorization and solving it with factorized gradient descent (FGD). It proves convergence and error guarantees under noise-free and noisy conditions, and in both exact-rank and over-parameterized regimes, while avoiding expensive $t$-SVD computations. Initialization does not require exact tubal-rank, and the method remains robust when $r$ slightly exceeds $r_{igstar}$. Empirical results show faster convergence and smaller recovery error than competing t-SVD–based methods, highlighting practical scalability and effectiveness for large-scale tensor data.

Abstract

This paper considers the problem of recovering a tensor with an underlying low-tubal-rank structure from a small number of corrupted linear measurements. Traditional approaches tackling such a problem require the computation of tensor Singular Value Decomposition (t-SVD), that is a computationally intensive process, rendering them impractical for dealing with large-scale tensors. Aim to address this challenge, we propose an efficient and effective low-tubal-rank tensor recovery method based on a factorization procedure akin to the Burer-Monteiro (BM) method. Precisely, our fundamental approach involves decomposing a large tensor into two smaller factor tensors, followed by solving the problem through factorized gradient descent (FGD). This strategy eliminates the need for t-SVD computation, thereby reducing computational costs and storage requirements. We provide rigorous theoretical analysis to ensure the convergence of FGD under both noise-free and noisy situations. Additionally, it is worth noting that our method does not require the precise estimation of the tensor tubal-rank. Even in cases where the tubal-rank is slightly overestimated, our approach continues to demonstrate robust performance. A series of experiments have been carried out to demonstrate that, as compared to other popular ones, our approach exhibits superior performance in multiple scenarios, in terms of the faster computational speed and the smaller convergence error.

Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent

TL;DR

The paper tackles low-tubal-rank tensor recovery from limited, noisy measurements by introducing a Burer-Monteiro–style tensor factorization and solving it with factorized gradient descent (FGD). It proves convergence and error guarantees under noise-free and noisy conditions, and in both exact-rank and over-parameterized regimes, while avoiding expensive -SVD computations. Initialization does not require exact tubal-rank, and the method remains robust when slightly exceeds . Empirical results show faster convergence and smaller recovery error than competing t-SVD–based methods, highlighting practical scalability and effectiveness for large-scale tensor data.

Abstract

This paper considers the problem of recovering a tensor with an underlying low-tubal-rank structure from a small number of corrupted linear measurements. Traditional approaches tackling such a problem require the computation of tensor Singular Value Decomposition (t-SVD), that is a computationally intensive process, rendering them impractical for dealing with large-scale tensors. Aim to address this challenge, we propose an efficient and effective low-tubal-rank tensor recovery method based on a factorization procedure akin to the Burer-Monteiro (BM) method. Precisely, our fundamental approach involves decomposing a large tensor into two smaller factor tensors, followed by solving the problem through factorized gradient descent (FGD). This strategy eliminates the need for t-SVD computation, thereby reducing computational costs and storage requirements. We provide rigorous theoretical analysis to ensure the convergence of FGD under both noise-free and noisy situations. Additionally, it is worth noting that our method does not require the precise estimation of the tensor tubal-rank. Even in cases where the tubal-rank is slightly overestimated, our approach continues to demonstrate robust performance. A series of experiments have been carried out to demonstrate that, as compared to other popular ones, our approach exhibits superior performance in multiple scenarios, in terms of the faster computational speed and the smaller convergence error.
Paper Structure (21 sections, 10 theorems, 88 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 21 sections, 10 theorems, 88 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main contributions
  3. Related work
  4. Low rank tensor recovery with t-SVD framework
  5. Low rank tensor recovery with other decomposition frameworks
  6. Notations and preliminaries
  7. Main results
  8. Formulation of tensor BM factorization
  9. Main Theorem
  10. Population-sample analysis
  11. Population analysis
  12. Finite sample analysis
  13. Main proofs
  14. Proof of Lemma 6
  15. Proof of Theorem 3
  16. ...and 6 more sections

Key Result

Theorem 1

Let $\mathcal{A}\in\mathbb{R}^{n_1\times n_2 \times n_3}$, then it can be factored as where $\mathcal{U}\in\mathbb{R}^{n_1 \times n_1 \times n_3}$, $\mathcal{V}\in\mathbb{R}^{n_2\times n_2 \times n_3}$ are orthogonal tensors, and $\mathcal{S}\in\mathbb{R}^{n_1\times n_2 \times n_3}$ is a f-diagonal tensor.

Figures (4)

  • Figure 1: Simulations that verify Lemmas 3 and 6. Subfigures (a) and (b) respectively depict the outcomes of employing FGD to solve problem (\ref{['equ:10']}) under the exact rank and over rank scenarios, which are the results of Lemma 3. Subfigures (c) and (d) respectively depict the outcomes of employing FGD to solve problem (\ref{['equ:7']}) under the exact rank and over rank scenarios, which are the results of Lemma 6. We set $n=50$, $n_3=5$, $r_{\star}=3$, $\textbf{s}=0$. For exact rank case, we set $r=r_{\star}$; for over rank case, we set $r=5$.
  • Figure 2: Phase transitions for LTRTR of TNN and our FGD method. We set $n=30$, $n_3=5$. The step size of FGD is set to be 0.001.
  • Figure 3: The relative error and clock time of three methods in the noiseless case. Subfigures (a) and (b) present a comparison of the convergence rate and computation time for three algorithms under the exact rank scenario. Subfigures (c) and (d) present a comparison of the convergence rate and computation time for three algorithms under the over rank scenario. We set $n=50,\ n_3=5,\ r_{\star}=3,\ m=10(2n-r_{\star})n_3,\ \eta=0.001$. For exact rank case, we set $r=r_{\star}$; for over rank case, we set $r=r_{\star}+2$. We stop when $\|\mathcal{X}_t-\mathcal{X}_{\star}\|_F/\|\mathcal{X}_{\star}\|_F\le 10^{-5}$ for exact rank case and $\|\mathcal{X}_t-\mathcal{X}_{\star}\|_F/\|\mathcal{X}_{\star}\|_F\le 10^{-2}$ for the over rank case.
  • Figure 4: The relative error and clock time of three methods in the noisy case. Subfigures (a) and (b) present a comparison of the convergence rate and computation time for three algorithms under the exact rank scenario. Subfigures (c) and (d) present a comparison of the convergence rate and computation time for three algorithms under the over rank scenario. We set $n=50,\ n_3=5,\ r_{\star}=3,\ m=10(2n-r_{\star})n_3,\ s_i\sim\mathcal{N}(0,0.5^2),\ \eta=0.001$. For exact rank case, we set $r=r_{\star}$; for over rank case, we set $r=r_{\star}+2$.

Theorems & Definitions (29)

  • Definition 2.1: Block diagonal matrix kilmer2011factorization
  • Definition 2.2: Block circulant matrix kilmer2011factorization
  • Definition 2.3: The fold and unfold operations kilmer2011factorization
  • Definition 2.4: T-productkilmer2011factorization
  • Definition 2.5: Tensor conjugate transposekilmer2011factorization
  • Definition 2.6: Identity tensorkilmer2011factorization
  • Definition 2.7: Orthogonal tensor kilmer2011factorization
  • Definition 2.8: F-diagonal tensor kilmer2011factorization
  • Theorem 1: t-SVD kilmer2011factorizationlu2018exact
  • Definition 2.9: Tubal-rank kilmer2011factorization
  • ...and 19 more