Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algorithm xxx: Faster Randomized SVD with Dynamic Shifts

Xu Feng, Wenjian Yu, Yuyang Xie, Jie Tang

TL;DR

This work tackles the challenge of efficiently computing a few leading singular values and vectors of large sparse matrices. It introduces dashSVD, a randomized SVD method that uses dynamically updated shifted power iterations to accelerate convergence while preserving accuracy, coupled with a per-vector error (PVE) based accuracy-control mechanism. The authors prove an error bound for the shifted power iteration and empirically show that dashSVD speeds up computations substantially over state-of-the-art solvers (e.g., LanczosBD and PRIMME_SVDS) for moderate accuracy, with comparable memory usage and strong robustness to matrices with slowly decaying singular values. The approach offers practical gains in runtime and parallel efficiency for real-world sparse data, with potential extensions to distributed-memory environments.

Abstract

Aiming to provide a faster and convenient truncated SVD algorithm for large sparse matrices from real applications (i.e. for computing a few of largest singular values and the corresponding singular vectors), a dynamically shifted power iteration technique is applied to improve the accuracy of the randomized SVD method. This results in a dynamic shifts based randomized SVD (dashSVD) algorithm, which also collaborates with the skills for handling sparse matrices. An accuracy-control mechanism is included in the dashSVD algorithm to approximately monitor the per vector error bound of computed singular vectors with negligible overhead. Experiments on real-world data validate that the dashSVD algorithm largely improves the accuracy of randomized SVD algorithm or attains same accuracy with fewer passes over the matrix, and provides an efficient accuracy-control mechanism to the randomized SVD computation, while demonstrating the advantages on runtime and parallel efficiency. A bound of the approximation error of the randomized SVD with the shifted power iteration is also proved.

Algorithm xxx: Faster Randomized SVD with Dynamic Shifts

TL;DR

This work tackles the challenge of efficiently computing a few leading singular values and vectors of large sparse matrices. It introduces dashSVD, a randomized SVD method that uses dynamically updated shifted power iterations to accelerate convergence while preserving accuracy, coupled with a per-vector error (PVE) based accuracy-control mechanism. The authors prove an error bound for the shifted power iteration and empirically show that dashSVD speeds up computations substantially over state-of-the-art solvers (e.g., LanczosBD and PRIMME_SVDS) for moderate accuracy, with comparable memory usage and strong robustness to matrices with slowly decaying singular values. The approach offers practical gains in runtime and parallel efficiency for real-world sparse data, with potential extensions to distributed-memory environments.

Abstract

Aiming to provide a faster and convenient truncated SVD algorithm for large sparse matrices from real applications (i.e. for computing a few of largest singular values and the corresponding singular vectors), a dynamically shifted power iteration technique is applied to improve the accuracy of the randomized SVD method. This results in a dynamic shifts based randomized SVD (dashSVD) algorithm, which also collaborates with the skills for handling sparse matrices. An accuracy-control mechanism is included in the dashSVD algorithm to approximately monitor the per vector error bound of computed singular vectors with negligible overhead. Experiments on real-world data validate that the dashSVD algorithm largely improves the accuracy of randomized SVD algorithm or attains same accuracy with fewer passes over the matrix, and provides an efficient accuracy-control mechanism to the randomized SVD computation, while demonstrating the advantages on runtime and parallel efficiency. A bound of the approximation error of the randomized SVD with the shifted power iteration is also proved.
Paper Structure (16 sections, 12 theorems, 21 equations, 10 figures, 5 tables, 5 algorithms)

This paper contains 16 sections, 12 theorems, 21 equations, 10 figures, 5 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Basics of Truncated SVD
  5. Randomized SVD Algorithm with Power Iteration
  6. Algorithms in svds and PRIMME_SVDS
  7. Randomized SVD with Dynamic Shifts
  8. Shifted Power Iteration and Setting Dynamic Shifts
  9. Accuracy Control Based on PVE Bound
  10. Performance Analysis
  11. Validation of the Shifted Power Iteration
  12. Validation of the dashSVD Algorithm
  13. Validation of the Accuracy-Control Scheme Based on PVE Criterion
  14. Conclusions
  15. The proof of Theorem 1
  16. ...and 1 more sections

Key Result

Lemma 1

Suppose $\mathbf{A},\mathbf{C}\in\mathbb{R}^{m\times n}$. The following inequalities hold for the decreasingly ordered singular values of $\mathbf{A}$, $\mathbf{C}$ and $\mathbf{AC}^\mathrm{T}$ ($1\le i, j, i+j -1 \le \min(m, n)$) and

Figures (10)

  • Figure 1: The PVE error vs. power parameter curves of three randomized SVD algorithms ($k=100$).
  • Figure 2: The error vs. time curves of dashSVD and LanczosBD in svds with single-thread computing ($k=100$). The unit of time is second.
  • Figure 3: The error vs. runtime curves of dashSVD and PRIMME_SVDS with 8-thread computing ($k=100$). The unit of time is second.
  • Figure 4: The error vs. time curves of dashSVD compared with LanczosBD in svds and PRIMME_SVDS for LargeRegFile ($k=100$). The unit of time is second.
  • Figure 5: Experimental results on LargeRegFile ($k=100$).
  • ...and 5 more figures

Theorems & Definitions (14)

  • Lemma 1
  • Proposition 1
  • Remark 1
  • Lemma 2
  • Lemma 3
  • Proposition 2
  • Theorem 1
  • Remark 2
  • Proposition 3
  • Lemma 4
  • ...and 4 more