Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On a randomized small-block Lanczos method for large-scale null space computations

Daniel Kressner, Nian Shao

TL;DR

The paper tackles large-scale null-space computation for sparse matrices by introducing a randomized small-block Lanczos method applied to $B = A^{\mathsf{T}}A + \varepsilon D$, where a small diagonal perturbation $\varepsilon D$ and a random initial guess enable reliable convergence even for $d=1$. It provides a perturbation theory that shows zero-eigenvalue repulsion and analyzes how the perturbation affects the null-space residual, followed by a self-contained convergence analysis for single-vector Krylov subspaces and Rayleigh–Ritz extraction. Practical enhancements including restarting, partial reorthogonalization, and preconditioning are developed, and a detailed algorithm with pseudocode is presented. Numerical experiments on graph connectivity and cohomology problems demonstrate memory and computational advantages over traditional null-space solvers and large-block Lanczos, particularly when the nullity is moderate and memory is constrained. The work establishes a solid theoretical understanding for $d=1$ and offers a scalable, efficient framework for incremental null-space computation in large-scale settings, with Open questions remaining for $d>1$ scenarios.

Abstract

Computing the null space of a large sparse matrix $A$ is a challenging computational problem, especially if the nullity -- the dimension of the null space -- is not small. When applying a block Lanczos method to $A^\mathsf{T} A$ for this purpose, conventional wisdom suggests to use a block size $d$ that is not smaller than the nullity. In this work, we show how randomness can be utilized to allow for smaller $d$ without sacrificing convergence or reliability. Even $d = 1$, corresponding to the standard single-vector Lanczos method, becomes a safe choice. This is achieved by using a small random diagonal perturbation, which moves the zero eigenvalues of $A^\mathsf{T} A$ away from each other, and a random initial guess. We analyze the effect of the perturbation on the attainable quality of the null space and derive convergence results that establish robust convergence for $d=1$. As demonstrated by our numerical experiments, a smaller block size combined with restarting and partial reorthogonalization results in reduced memory requirements and computational effort. It also allows for the incremental computation of the null space, without requiring a priori knowledge of the nullity. Our algorithm is best suited for situations when the nullity of $A$ is moderate.

On a randomized small-block Lanczos method for large-scale null space computations

TL;DR

The paper tackles large-scale null-space computation for sparse matrices by introducing a randomized small-block Lanczos method applied to , where a small diagonal perturbation and a random initial guess enable reliable convergence even for . It provides a perturbation theory that shows zero-eigenvalue repulsion and analyzes how the perturbation affects the null-space residual, followed by a self-contained convergence analysis for single-vector Krylov subspaces and Rayleigh–Ritz extraction. Practical enhancements including restarting, partial reorthogonalization, and preconditioning are developed, and a detailed algorithm with pseudocode is presented. Numerical experiments on graph connectivity and cohomology problems demonstrate memory and computational advantages over traditional null-space solvers and large-block Lanczos, particularly when the nullity is moderate and memory is constrained. The work establishes a solid theoretical understanding for and offers a scalable, efficient framework for incremental null-space computation in large-scale settings, with Open questions remaining for scenarios.

Abstract

Computing the null space of a large sparse matrix is a challenging computational problem, especially if the nullity -- the dimension of the null space -- is not small. When applying a block Lanczos method to for this purpose, conventional wisdom suggests to use a block size that is not smaller than the nullity. In this work, we show how randomness can be utilized to allow for smaller without sacrificing convergence or reliability. Even , corresponding to the standard single-vector Lanczos method, becomes a safe choice. This is achieved by using a small random diagonal perturbation, which moves the zero eigenvalues of away from each other, and a random initial guess. We analyze the effect of the perturbation on the attainable quality of the null space and derive convergence results that establish robust convergence for . As demonstrated by our numerical experiments, a smaller block size combined with restarting and partial reorthogonalization results in reduced memory requirements and computational effort. It also allows for the incremental computation of the null space, without requiring a priori knowledge of the nullity. Our algorithm is best suited for situations when the nullity of is moderate.
Paper Structure (27 sections, 12 theorems, 68 equations, 1 figure, 7 tables)

This paper contains 27 sections, 12 theorems, 68 equations, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Assumptions and notation
  3. Basic algorithm
  4. Perturbation analysis
  5. Perturbation and repulsion of zero eigenvalues
  6. Perturbation of null space residual
  7. Principal angles and null space residual
  8. Convergence analysis
  9. Convergence of single-vector Krylov subspaces for small eigenvalues
  10. The case $\ell = t$. Proof of \ref{['eqLan1']}
  11. The case $\ell > t$. Proof of \ref{['eqLan2']}
  12. Convergence of \ref{['algo']} with a single initial vector
  13. Practical techniques for implementation
  14. Restarting
  15. Partial reorthogonalization
  16. ...and 12 more sections

Key Result

Proposition 2.1

\newlabelrepulsion0 Let $B_0 \in\mathbb{R}^{n\times n}$ be symmetric, and let $D\in\mathbb{R}^{n\times n}$ be a random diagonal matrix with the diagonal entries drawn i.i.d. from a distribution with bounded probability density function $\rho(\cdot)$. Then for any bounded interval $\mathcal{I}\subs holds for some universal constant $C_0$, where $\lvert\mathcal{I}\rvert$ is the length of $\mathcal{

Figures (1)

  • Figure 1: Convergence history of single-vector Lanczos with $\epsilon=4^{-10}$ for graph Laplacian: Number of converged Ritz values vs. number of matrix-vector multiplications.

Theorems & Definitions (25)

  • Remark 1.1
  • Remark 1.2
  • Proposition 2.1: Equation (1.11) in Aizenman2017 attributed to Minami1996
  • Theorem 2.2
  • Proof 1
  • Lemma 2.3
  • Proof 2
  • Theorem 2.4
  • Proof 3
  • Definition 2.5
  • ...and 15 more