Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A multilinear Nyström algorithm for low-rank approximation of tensors in Tucker format

Alberto Bucci, Leonardo Robol

TL;DR

This work extends randomized low-rank approximation to tensors by introducing the multilinear Nyström (MLN) method for Tucker decomposition. MLN generalizes the generalized Nyström approach to higher orders using oblique projections and per-mode sketches, achieving near-optimal accuracy with a single-pass, streamable scheme; stability is ensured through an epsilon-pseudoinverse variant (SMLN) and extra sketching. The authors derive deterministic and probabilistic accuracy bounds linked to mode-wise sketches, and provide stability analyses under finite-precision arithmetic, demonstrating practical performance gains over traditional HOSVD-based methods with favorable memory and data-access profiles. Empirical results corroborate the theory, showing near-RHOSVD accuracy with modest oversampling and manageable computation times, while also highlighting the benefits of structured sketching and the limitations as tensor order grows. Overall, MLN offers a scalable, stable alternative for large-scale tensor compression and can be integrated with structured-tensor formats and related tensor-network approaches.

Abstract

The Nyström method offers an effective way to obtain low-rank approximation of SPD matrices, and has been recently extended and analyzed to nonsymmetric matrices (leading to the generalized Nyström method). It is a randomized, single-pass, streamable, cost-effective, and accurate alternative to the randomized SVD, and it facilitates the computation of several matrix low-rank factorizations. In this paper, we take these advancements a step further by introducing a higher-order variant of Nyström's methodology tailored to approximating low-rank tensors in the Tucker format: the multilinear Nyström technique. We show that, by introducing appropriate small modifications in the formulation of the higher-order method, strong stability properties can be obtained. This algorithm retains the key attributes of the generalized Nyström method, positioning it as a viable substitute for the randomized higher-order SVD algorithm.

A multilinear Nyström algorithm for low-rank approximation of tensors in Tucker format

TL;DR

This work extends randomized low-rank approximation to tensors by introducing the multilinear Nyström (MLN) method for Tucker decomposition. MLN generalizes the generalized Nyström approach to higher orders using oblique projections and per-mode sketches, achieving near-optimal accuracy with a single-pass, streamable scheme; stability is ensured through an epsilon-pseudoinverse variant (SMLN) and extra sketching. The authors derive deterministic and probabilistic accuracy bounds linked to mode-wise sketches, and provide stability analyses under finite-precision arithmetic, demonstrating practical performance gains over traditional HOSVD-based methods with favorable memory and data-access profiles. Empirical results corroborate the theory, showing near-RHOSVD accuracy with modest oversampling and manageable computation times, while also highlighting the benefits of structured sketching and the limitations as tensor order grows. Overall, MLN offers a scalable, stable alternative for large-scale tensor compression and can be integrated with structured-tensor formats and related tensor-network approaches.

Abstract

The Nyström method offers an effective way to obtain low-rank approximation of SPD matrices, and has been recently extended and analyzed to nonsymmetric matrices (leading to the generalized Nyström method). It is a randomized, single-pass, streamable, cost-effective, and accurate alternative to the randomized SVD, and it facilitates the computation of several matrix low-rank factorizations. In this paper, we take these advancements a step further by introducing a higher-order variant of Nyström's methodology tailored to approximating low-rank tensors in the Tucker format: the multilinear Nyström technique. We show that, by introducing appropriate small modifications in the formulation of the higher-order method, strong stability properties can be obtained. This algorithm retains the key attributes of the generalized Nyström method, positioning it as a viable substitute for the randomized higher-order SVD algorithm.
Paper Structure (12 sections, 7 theorems, 68 equations, 7 figures, 2 tables, 3 algorithms)

This paper contains 12 sections, 7 theorems, 68 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary concepts and notation
  3. Randomized matrix low-rank approximation
  4. Multilinear Nyström
  5. Introducing more sketching matrices
  6. Properties of MLN
  7. Accuracy of MLN
  8. Accuracy of SMLN
  9. Stability of SMLN
  10. Sketch selection for MLN
  11. Experiments
  12. Conclusions

Key Result

Lemma 5.1

\newlabellemma_10 Let $\mathcal{A}$ be a $d$-dimensional tensor, and $\mathcal{P}_k := \mathcal{A}_k X_k(Y_k^T\mathcal{A}_k X_k)^{\dagger}Y_k^T$ for sketching matrices $X_k, Y_k$ of compatible dimensions. Then, the following inequality holds

Figures (7)

  • Figure 1: Accuracy of multilinear Nyström (MLN) and stabilized multilinear Nyström with $\epsilon = u\|A\|_F$ (SMLN-1) and $\epsilon = 10u\|A\|_F$ (SMLN-10).
  • Figure 2: Performance comparison of MLN with varying values of the oversampling parameter $\ell$ on two tensors of size $100\times 100\times 100$: one with $\sigma_i = 0.7^i$ (left) and another with $\sigma_i = 1/i$ (right)."
  • Figure 3: Comparison of MLN with oversampling parameter $\ell=r/2$, HOSVD, RHOSVD, and RSTHOSVD on tensors with different decays.
  • Figure 4: Frobenius error of approximation of the 3D Hilbert tensor: $\mathcal{H}(i,j,k) = \frac{1}{i+j+k-2}$ (left) and 4D Hilbert tensor: $\mathcal{H}(i,j,k,\ell) = \frac{1}{i+j+k+ \ell-3}$ (right).
  • Figure 5: Comparison of Tucker approximation methods in terms of computing time on 3D tensors (left) and 4D tensors (right) of fixed size.
  • ...and 2 more figures

Theorems & Definitions (16)

  • Definition 2.1
  • Lemma 5.1
  • Proof 1
  • Lemma 5.2
  • Proof 2
  • Lemma 5.3
  • Proof 3
  • Remark 5.4
  • Theorem 5.5: Deterministic accuracy bound
  • Proof 4
  • ...and 6 more