Randomized algorithms for distributed computation of principal component analysis and singular value decomposition
Huamin Li, Yuval Kluger, Mark Tygert
TL;DR
The paper develops randomized distributed algorithms for principal component analysis and singular value decomposition, addressing thin SVD for tall-and-skinny matrices and low-rank approximations of general matrices. It introduces two families of methods for tall-and-skinny data (a randomized TSVD with TSQR and a Gram-based variant) and two pipelines for general matrices (subspace-iteration and direct SVD variants), emphasizing double orthonormalization to achieve near machine-precision results. Comparisons against Spark MLlib show superior reconstruction accuracy and competitive runtimes, with practical guidance on when to prefer each approach and how to integrate these methods into distributed pipelines. Appendices demonstrate scalability with executor counts and provide test-matrix generation timings, reinforcing the methods' robustness and applicability to large-scale distributed computation.
Abstract
Randomized algorithms provide solutions to two ubiquitous problems: (1) the distributed calculation of a principal component analysis or singular value decomposition of a highly rectangular matrix, and (2) the distributed calculation of a low-rank approximation (in the form of a singular value decomposition) to an arbitrary matrix. Carefully honed algorithms yield results that are uniformly superior to those of the stock, deterministic implementations in Spark (the popular platform for distributed computation); in particular, whereas the stock software will without warning return left singular vectors that are far from numerically orthonormal, a significantly burnished randomized implementation generates left singular vectors that are numerically orthonormal to nearly the machine precision.