Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Sparse Singular Value Decomposition for High-dimensional High-order Data

Anru Zhang, Rungang Han

Abstract

In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure features a novel double projection \& thresholding scheme, which provides a sharp criterion for thresholding in each iteration. Compared with regular tensor SVD model, STAT-SVD permits more robust estimation under weaker assumptions. Both the upper and lower bounds for estimation accuracy are developed. The proposed procedure is shown to be minimax rate-optimal in a general class of situations. Simulation studies show that STAT-SVD performs well under a variety of configurations. We also illustrate the merits of the proposed procedure on a longitudinal tensor dataset on European country mortality rates.

Optimal Sparse Singular Value Decomposition for High-dimensional High-order Data

Abstract

In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure features a novel double projection \& thresholding scheme, which provides a sharp criterion for thresholding in each iteration. Compared with regular tensor SVD model, STAT-SVD permits more robust estimation under weaker assumptions. Both the upper and lower bounds for estimation accuracy are developed. The proposed procedure is shown to be minimax rate-optimal in a general class of situations. Simulation studies show that STAT-SVD performs well under a variety of configurations. We also illustrate the merits of the proposed procedure on a longitudinal tensor dataset on European country mortality rates.

Paper Structure

This paper contains 23 sections, 15 theorems, 120 equations, 6 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Notations and Preliminaries
  4. Sparse Tensor SVD Model
  5. The STAT-SVD Procedure
  6. Theoretical Analysis
  7. Data-driven Hyperparameter Selection
  8. Numerical Analysis
  9. Simulation Studies
  10. Mortality Rate Data Analysis
  11. Proofs
  12. Proof of Theorem \ref{['th:upper_bound']}
  13. Comparison with Single Thresholding & Projection Scheme
  14. Implementation Details of S-HOSVD and S-HOOI
  15. Additional Proofs
  16. ...and 8 more sections

Key Result

Theorem 1

Suppose ${\boldsymbol{r}}, \sigma^2$ are known, $\log{p_1} \asymp \cdots \asymp \log{p_d}$, and for $1\leq k \leq d$, $\lambda_k = \sigma_{r_k}\left(\mathcal{M}_k({\mathbf{X}})\right) \geq C_0\sigma\left((s \log p)^{1/2} + \sum_{k} s_k r_k + \max_{1\leq k\leq d} r_{-k}\right).$ Then after at most $O with probability at least $1 - \frac{C(p_1+\cdots+p_d)\log(s\log p)}{p_1\cdots p_d}$. Here $C>0$ is

Figures (6)

  • Figure 1: Illustration of sparse tensor SVD model. Here, ${\mathbf{X}}$ is sparse along Modes-1 and -3.
  • Figure 3: Estimation error of $U_k$ (left panel) and ${\mathbf{X}}$ (right panel) for different methods. Here, $p_{1}=p_{2}=p_{3}=50$, $s_{1}=s_{2}=s_{3}=15$, $\sigma=1$, $\lambda=70$, and $r_{1}=r_{2}=r_{3}=r$ vary.
  • Figure 5: Estimation error of $U_k$ (left panel) and ${\mathbf{X}}$ (right panel) with uniform distributed noise.
  • Figure 6: Estimation error of $U_1$ (left top), $U_2$ (right top), $U_3$ (left bottom) and ${\mathbf{X}}$ (right bottom) in partial sparse setting. Here, $\lambda=60$, $p_{1}=p_{2}=p_{3}=50$, $s_{3}=p_{3}$, $s_{1}=s_{2}=20$, $r_{1}=r_{2}=r_{3}=10$.
  • Figure 7: Mortality rate data: $\hat{u}_{1}$, $\hat{u}_2$
  • ...and 1 more figures

Theorems & Definitions (20)

  • Remark 1
  • Theorem 1: Upper bound
  • Remark 2
  • Remark 3: Proof sketch for Theorem \ref{['th:upper_bound']}
  • Remark 4: Performance of Single Projection & Truncation Scheme
  • Theorem 2: Lower Bound: Subspace Estimation
  • Theorem 3: Lower bound: Tensor Recovery
  • Proposition 1: Concentration Inequality of $\hat{\sigma}^2$
  • Proposition 2
  • Theorem 4
  • ...and 10 more