Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analysis of randomized CholeskyQR for sparse matrices

Haoran Guan, Yuwei Fan

TL;DR

This work analyzes rounding errors in randomized CholeskyQR-type QR factorizations for sparse matrices and introduces a novel sparse-matrix model that distinguishes dense and sparse columns. It develops a theoretical rounding-error framework for RCholeskyQR2 and related randomized methods under Gaussian sketching and (epsilon,p,n) oblivious l2-subspace embeddings, proving high-probability bounds on orthogonality and residual. The results show that randomized CholeskyQR-type algorithms can be applicable to more ill-conditioned sparse inputs while retaining accuracy comparable to deterministic counterparts, with distinct phenomena appearing in the sparse regime. Numerical experiments corroborate the theory, reveal sparse-specific behavior, and demonstrate the methods’ applicability, efficiency, and robustness in practice.

Abstract

This work is about rounding error analysis of randomized CholeskyQR-type algorithms for sparse matrices. We often encounter QR factorization of the sparse matrices in many real problems. In this work, we focus on some typical CholeskyQR-type algorithms with matrix sketching, which is a popular randomized technique in recent years. We build a new model of the sparse matrices and provide rounding error analysis of randomized CholeskyQR-type algorithms for the sparse cases with this model. We make comparison between the bounds with different models of sparsity both theoretically and experimentally. Numerical experiments show some new phenomena of randomized CholeskyQR-type algorithms for the sparse cases, which do not occur in the common sparse cases. We also test the applicability, accuracy, efficiency and robustness of randomized CholeskyQR-type algorithms for sparse matrices.

Analysis of randomized CholeskyQR for sparse matrices

TL;DR

This work analyzes rounding errors in randomized CholeskyQR-type QR factorizations for sparse matrices and introduces a novel sparse-matrix model that distinguishes dense and sparse columns. It develops a theoretical rounding-error framework for RCholeskyQR2 and related randomized methods under Gaussian sketching and (epsilon,p,n) oblivious l2-subspace embeddings, proving high-probability bounds on orthogonality and residual. The results show that randomized CholeskyQR-type algorithms can be applicable to more ill-conditioned sparse inputs while retaining accuracy comparable to deterministic counterparts, with distinct phenomena appearing in the sparse regime. Numerical experiments corroborate the theory, reveal sparse-specific behavior, and demonstrate the methods’ applicability, efficiency, and robustness in practice.

Abstract

This work is about rounding error analysis of randomized CholeskyQR-type algorithms for sparse matrices. We often encounter QR factorization of the sparse matrices in many real problems. In this work, we focus on some typical CholeskyQR-type algorithms with matrix sketching, which is a popular randomized technique in recent years. We build a new model of the sparse matrices and provide rounding error analysis of randomized CholeskyQR-type algorithms for the sparse cases with this model. We make comparison between the bounds with different models of sparsity both theoretically and experimentally. Numerical experiments show some new phenomena of randomized CholeskyQR-type algorithms for the sparse cases, which do not occur in the common sparse cases. We also test the applicability, accuracy, efficiency and robustness of randomized CholeskyQR-type algorithms for sparse matrices.

Paper Structure

This paper contains 33 sections, 14 theorems, 81 equations, 18 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Development of CholeskyQR
  3. Our considerations and contributions
  4. Outline of this work and notations
  5. Some theoretical results from the literature
  6. Lemmas for rounding error analysis
  7. Matrix sketching
  8. Theoretical results of some existing CholeskyQR-type algorithms
  9. Rounding error analysis of randomized CholeskyQR for sparse matrices
  10. The general structure of randomized CholeskyQR
  11. A new model of the sparse matrices
  12. RCholeskyQR2 for sparse matrices
  13. Settings of RCholeskyQR2 for sparse matrices
  14. Theoretical results of RCholeskyQR2 for sparse matrices
  15. Lemmas for proving Theorem \ref{['theorem 3.3']}
  16. ...and 18 more sections

Key Result

Lemma 2.1

If $A,B \in \mathbb{R}^{m\times n}$, then

Theorems & Definitions (24)

  • Lemma 2.1: Weyl's Theorem for singular values
  • Lemma 2.2: Rounding error in matrix multiplications
  • Lemma 2.3: Rounding error in Cholesky factorization
  • Definition 2.4: $\epsilon$-subspace embedding
  • Definition 2.5: $(\epsilon,p,n)$ oblivious $l_{2}$-subspace embedding
  • Lemma 2.6: Some properties of matrix sketching
  • Lemma 2.7: Rounding error analysis of CholeskyQR2
  • Lemma 2.8: Rounding error analysis of RHC
  • Definition 3.1: The new model of the sparse matrices based on the division of columns
  • Theorem 3.3: Rounding error analysis of RCholeskyQR2 for sparse matrices
  • ...and 14 more