Randomized algorithms for low-rank matrix approximation: Design, analysis, and applications
Joel A. Tropp, Robert J. Webber
TL;DR
This survey compares randomized low-rank matrix approximation methods—RSVD, RSI, and RBKI—and their Nyström variants, tying algorithmic choices to the singular-value structure of the input and available resources. It delivers new, explicit error bounds, improved RBKI pseudocode, and practical recommendations, including novel NysBKI variants for psd matrices. The work demonstrates that RSVD and NysSVD excel on rapidly decaying spectra, while RBKI and NysBKI offer superior accuracy and efficiency on challenging, slowly decaying spectra, with impactful applications in genetics PCA and kernel spectral clustering for molecular dynamics. Collectively, the results support broader adoption of Krylov-based randomized methods in computational science by providing clear guidelines, robust theory, and scalable implementations.
Abstract
This survey explores modern approaches for computing low-rank approximations of high-dimensional matrices by means of the randomized SVD, randomized subspace iteration, and randomized block Krylov iteration. The paper compares the procedures via theoretical analyses and numerical studies to highlight how the best choice of algorithm depends on spectral properties of the matrix and the computational resources available. Despite superior performance for many problems, randomized block Krylov iteration has not been widely adopted in computational science. The paper strengthens the case for this method in three ways. First, it presents new pseudocode that can significantly reduce computational costs. Second, it provides a new analysis that yields simple, precise, and informative error bounds. Last, it showcases applications to challenging scientific problems, including principal component analysis for genetic data and spectral clustering for molecular dynamics data.