Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

Oleg Balabanov, Laura Grigori

TL;DR

This work presents RBGS, a randomized block Gram-Schmidt procedure for efficient and stable QR factorization that extends single-vector RGS to a block setting. RBGS leverages random sketching to form a sketched inner product, enabling low-communication, cache-friendly, and parallelizable orthogonalization, with a Cholesky-QR augmentation optionally providing near-machine-precision orthogonality. The authors establish rigorous a priori and a posteriori stability guarantees, including an explicit epsilon's embedding framework, and demonstrate RBGS-based randomized block Arnoldi, GMRES, FOM, and RR methods for solving linear systems and eigenvalue problems, along with numerical experiments showing robust performance on rank-deficient and clustered problems. The approach offers substantial reductions in synchronization and data movement while maintaining accuracy, making it attractive for large-scale, high-performance computing environments and low-precision architectures. Potential extensions include model-order reduction, s-step Krylov methods, and broader applicability to orthogonalization-centric algorithms.

Abstract

This article introduces randomized block Gram-Schmidt process (RBGS) for QR decomposition. RBGS extends the single-vector randomized Gram-Schmidt (RGS) algorithm and inherits its key characteristics such as being more efficient and having at least as much stability as any deterministic (block) Gram-Schmidt algorithm. Block algorithms offer superior performance as they are based on BLAS3 matrix-wise operations and reduce communication cost when executed in parallel. Notably, our low-synchronization variant of RBGS can be implemented in a parallel environment using only one global reduction operation between processors per block. Moreover, the block Gram-Schmidt orthogonalization is the key element in the block Arnoldi procedure for the construction of a Krylov basis, which in turn is used in GMRES, FOM and Rayleigh-Ritz methods for the solution of linear systems and clustered eigenvalue problems. In this article, we develop randomized versions of these methods, based on RBGS, and validate them on nontrivial numerical examples.

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems

TL;DR

This work presents RBGS, a randomized block Gram-Schmidt procedure for efficient and stable QR factorization that extends single-vector RGS to a block setting. RBGS leverages random sketching to form a sketched inner product, enabling low-communication, cache-friendly, and parallelizable orthogonalization, with a Cholesky-QR augmentation optionally providing near-machine-precision orthogonality. The authors establish rigorous a priori and a posteriori stability guarantees, including an explicit epsilon's embedding framework, and demonstrate RBGS-based randomized block Arnoldi, GMRES, FOM, and RR methods for solving linear systems and eigenvalue problems, along with numerical experiments showing robust performance on rank-deficient and clustered problems. The approach offers substantial reductions in synchronization and data movement while maintaining accuracy, making it attractive for large-scale, high-performance computing environments and low-precision architectures. Potential extensions include model-order reduction, s-step Krylov methods, and broader applicability to orthogonalization-centric algorithms.

Abstract

This article introduces randomized block Gram-Schmidt process (RBGS) for QR decomposition. RBGS extends the single-vector randomized Gram-Schmidt (RGS) algorithm and inherits its key characteristics such as being more efficient and having at least as much stability as any deterministic (block) Gram-Schmidt algorithm. Block algorithms offer superior performance as they are based on BLAS3 matrix-wise operations and reduce communication cost when executed in parallel. Notably, our low-synchronization variant of RBGS can be implemented in a parallel environment using only one global reduction operation between processors per block. Moreover, the block Gram-Schmidt orthogonalization is the key element in the block Arnoldi procedure for the construction of a Krylov basis, which in turn is used in GMRES, FOM and Rayleigh-Ritz methods for the solution of linear systems and clustered eigenvalue problems. In this article, we develop randomized versions of these methods, based on RBGS, and validate them on nontrivial numerical examples.
Paper Structure (35 sections, 10 theorems, 129 equations, 3 figures, 11 algorithms)

This paper contains 35 sections, 10 theorems, 129 equations, 3 figures, 11 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. BGS process
  4. Random sketching
  5. Effect of random sketching on rounding errors
  6. RBGS process
  7. Solution of block least-squares problem in step 2
  8. Inter-block QR factorization in steps 4--5
  9. Single-vector RGS algorithm
  10. RCholeskyQR factorization
  11. Implicit RCholeskyQR factorization
  12. $\ell_2$-QR+RCholeskyQR factorization
  13. Stability analysis
  14. Assumptions
  15. Stability guarantees of RBGS algorithm
  16. ...and 20 more sections

Key Result

Corollary 1.3

If $\mathbf{\Theta} \in \mathbb{R}^{k \times n}$ is a $(\varepsilon, \delta/n, 1)$ oblivious $\ell_2$-subspace embedding, then with probability at least $1-\delta$, we have

Figures (3)

  • Figure 1: Block QR factorization of the matrix from balabanov2021randomizedGS. In the plots, "u.p. CG-RBGS" and "m.p. CG-RBGS" refer to the unique-precision and multi-precision RBGS algorithms, respectively, both of which perform step 2 with $20$ iterations of CG. On the other hand, "u.p. HH-RBGS" refers to the unique precision RBGS that uses a Householder solver.
  • Figure 2: Solution of a linear system with GMRES.
  • Figure 3: Solution of an eigenvalue problem with the RR method. In the plots, "BCGS-restarted" refers to the BCGS-Arnoldi algorithm that restarts every $5$ iterations.

Theorems & Definitions (27)

  • Definition 1.1
  • Definition 1.2
  • Corollary 1.3
  • Corollary 1.4
  • Corollary 1.5
  • Remark 2.1
  • Remark 2.3: Relation between RBGS of $\mathbf{W}$ and RCholeskyQR of $\mathbf{W}$
  • Theorem 3.2
  • proof
  • Remark 3.3
  • ...and 17 more