Randomized Algorithms for Low-Rank Matrix and Tensor Decompositions

TL;DR

This survey compiles the state of randomized numerical linear algebra for low-rank matrix and tensor decompositions, linking classic matrix algorithms with modern randomized dimension reduction. It systematically covers foundations (SVD, QR, ID/CUR), randomized rangefinding and SVD, and tensor extensions to CP and Tucker formats using sketching and sampling, including TensorSketch and leverage-score methods. The work emphasizes error estimation, certificates, and adaptive strategies, and highlights practical gains in scalability, parallelism, and hardware acceleration. By connecting matrix RNLA with tensor decompositions, it provides a cohesive view of fast, structure-preserving approaches for large-scale data processing and outlines avenues for theory, software, and future tensor methods.

Abstract

This paper surveys randomized algorithms in numerical linear algebra for low-rank decompositions of matrices and tensors. The survey begins with a review of classical matrix algorithms that can be accelerated by randomized dimensionality reduction, such as the singular value decomposition (SVD) or interpolative (ID) and CUR decompositions. Recent advances in randomized dimensionality reduction are discussed, including new methods of fast matrix sketching and sampling techniques, which are incorporated into classical matrix algorithms for fast low-rank matrix approximations. The extension of randomized matrix algorithms to tensors is then explored for several low-rank tensor decompositions in the CP and Tucker formats, including the higher-order SVD, ID, and CUR decomposition.

Figures (2)

• Figure 1: Rank-$R$ CP decomposition $\tx = \sum_{r=1}^R \xa_r \circ \xb_r \circ \xc_r$ for 3-mode tensor $\tx \in \field^{N_1 \times N_2 \times N_3}$.
• Figure 2: Rank-$(R_1,R_2,R_3)$ Tucker decomposition $\tx = \tg \times_1 \ma^{(1)} \times_2 \ma^{(2)} \times_3 \ma^{(3)}$ of $\tx \in \field^{N_1 \times N_2 \times N_3}$.