Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast, High-Accuracy, Randomized Nullspace Computations for Tall Matrices

Ethan N. Epperly, Taejun Park, Yuji Nakatsukasa

TL;DR

RLOBPCG tackles the problem of efficiently computing a small number of minimum singular vectors for very tall matrices by marrying randomized sketch-based preconditioning with the LOBPCG eigensolver applied to $A^*A$. Under a subspace-embedding distortion $\eta$ and a modest singular-value gap, the method converges geometrically to the minimum right singular vector, with practical runtimes supported by a rigorous complexity bound. Empirical results demonstrate near-optimal accuracy up to $10^6$ rows, outperforming classical LOBPCG and Lanczos methods by up to $12\times$ and maintaining robustness where other methods fail. A block variant extends the approach to subspaces, improving reliability for tight gaps and enabling simultaneous computation of multiple singular triplets. The work also shows how RLOBPCG can accelerate rational approximation tasks (AAA-Lawson) that rely on solving near-nullspace problems, highlighting its practical impact in large-scale numerical linear algebra.

Abstract

In this paper, we develop RLOBPCG, an efficient method for computing a small number of singular triplets corresponding to the smallest singular values of large, tall matrices. The algorithm combines randomized preconditioner from the sketch-and-precondition techniques with the LOBPCG eigensolver: a small sketch is used to construct a high-quality preconditioner, and LOBPCG is run on the Gram matrix to refine the singular vector. Under the standard subspace embedding assumption and a modest singular value gap between the two smallest singular values, we prove that RLOBPCG converges geometrically to the minimum singular vector. In numerical experiments, RLOBPCG achieves near-optimal accuracy on matrices with up to $10^6$ rows, outperforming classical LOBPCG and Lanczos methods by a speedup of up to $12\times$ and maintaining robustness when other iterative methods fail to converge.

Fast, High-Accuracy, Randomized Nullspace Computations for Tall Matrices

TL;DR

RLOBPCG tackles the problem of efficiently computing a small number of minimum singular vectors for very tall matrices by marrying randomized sketch-based preconditioning with the LOBPCG eigensolver applied to . Under a subspace-embedding distortion and a modest singular-value gap, the method converges geometrically to the minimum right singular vector, with practical runtimes supported by a rigorous complexity bound. Empirical results demonstrate near-optimal accuracy up to rows, outperforming classical LOBPCG and Lanczos methods by up to and maintaining robustness where other methods fail. A block variant extends the approach to subspaces, improving reliability for tight gaps and enabling simultaneous computation of multiple singular triplets. The work also shows how RLOBPCG can accelerate rational approximation tasks (AAA-Lawson) that rely on solving near-nullspace problems, highlighting its practical impact in large-scale numerical linear algebra.

Abstract

In this paper, we develop RLOBPCG, an efficient method for computing a small number of singular triplets corresponding to the smallest singular values of large, tall matrices. The algorithm combines randomized preconditioner from the sketch-and-precondition techniques with the LOBPCG eigensolver: a small sketch is used to construct a high-quality preconditioner, and LOBPCG is run on the Gram matrix to refine the singular vector. Under the standard subspace embedding assumption and a modest singular value gap between the two smallest singular values, we prove that RLOBPCG converges geometrically to the minimum singular vector. In numerical experiments, RLOBPCG achieves near-optimal accuracy on matrices with up to rows, outperforming classical LOBPCG and Lanczos methods by a speedup of up to and maintaining robustness when other iterative methods fail to converge.
Paper Structure (26 sections, 2 theorems, 41 equations, 5 figures, 2 algorithms)

This paper contains 26 sections, 2 theorems, 41 equations, 5 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Illustrative example
  3. Summary of theoretical results
  4. Roadmap
  5. Algorithms and implementation
  6. RLOBPCG implementation
  7. Orthogonalization and minimizing matrix--vector products
  8. Stopping criteria
  9. Choice of embedding
  10. Block version
  11. Theory
  12. Background: Results for sketching
  13. Background: Preconditioned eigensolvers
  14. Interpreting \ref{['eq:lobpcg_iteration_bound']}
  15. Proof of \ref{['thm:rlobpcg_convergence']}
  16. ...and 11 more sections

Key Result

Theorem 1.2

\newlabelthm:rlobpcg_convergence0 Let $\boldsymbol{A} \in \mathbb{C}^{m\times n}$ be a tall matrix with singular values $\sigma_1,\ldots,\sigma_n > 0$, and let $\boldsymbol{S} \in \mathbb{C}^{d\times m}$ be a subspace embedding for $\boldsymbol{A}$ with distortion $\eta \in [0,1)$. Introduce the r Suppose that the distortion satisfies Then $\boldsymbol{v}_k$, the approximation to $\boldsymbol{v}

Figures (5)

  • Figure 1: Relative singular value error $(\mleft\| \boldsymbol{A}\boldsymbol{v}_k \mright\| - \sigma_n) / \sigma_n$ vs. wall-clock time for iterative minimum singular value algorithms on easy (left) and hard (right) problems.
  • Figure 1: Convergence behavior of singular values (left) and singular vectors (right) of RLOBPCG on matrices with different singular value gaps. Dashed lines show predictions from matrix perturbation theory.
  • Figure 1: Error curve $r-f$ of rational approximation to $f(z)=z\operatorname{sign}(\mathrm{Re}(z))$ for AAA (top left) and AAA-Lawson with MATLAB's svd (bottom left), sketch-and-solve (top right), and RLOBPCG (bottom right).
  • Figure 2: Convergence behavior of singular values (left) and singular vectors (right) of RLOBPCG on matrices with different condition numbers. Dashed lines show predictions from matrix perturbation theory.
  • Figure 3: Singular value (left) and singular vector (right) error as a function of iteration count (top), total number of matrix-vector products (middle), and wall-clock time (bottom) for RLOBPCG with various block sizes. Dashed lines show predictions from matrix perturbation theory.

Theorems & Definitions (3)

  • Definition 1.1: Subspace embedding
  • Theorem 1.2: RLOBPCG: Convergence
  • Corollary 1.3: RLOBPCG: Runtime