Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Matrix Factorizations with Uniformly Random Pivoting

Isabel Detherage, Rikhav Shah

TL;DR

This work unifies two foundational families of matrix factorizations—the Jacobi eigenvalue algorithm and Gram-Schmidt—with Gaussian elimination/QR through a generalized factorization framework that uses randomized pivoting. It proves that, in expectation, all special cases converge linearly at the same rate independent of the targeted factorization, and it provides a robust finite-precision stability theory for orthogonalization, Jacobi, and related decompositions. The analysis introduces a Gamma-based potential that yields explicit contraction under random size-$k$ pivoting, and extends stability results to eigenvalues and singular values via existing bounds adapted to the randomized setting. Collectively, the results offer rigorous guarantees for both the convergence and numerical stability of a broad class of factorization algorithms, with practical implications for SVD/QR computations and block-structured variants. The framework also encompasses the Kaczmarz–Kac walk and supports block, adaptive, fixed, and randomized pivoting schemes, highlighting broad applicability across numerical linear algebra.

Abstract

This paper highlights a formal connection between two families of widely used matrix factorization algorithms in numerical linear algebra. One family consists of the Jacobi eigenvalue algorithm and its variants for computing the Hermitian eigendecomposition and singular value decomposition. The other consists of Gaussian elimination and the Gram-Schmidt procedure with various pivoting rules for computing the Cholesky decomposition and QR decomposition respectively. Both families are cast as special cases of a more general class of factorization algorithms. We provide a randomized pivoting rule that applies to this general class (which differs substantially from the usual pivoting rules for Gaussian elimination / Gram-Schmidt) which admits a unified analysis of the entire class of algorithms. The result is the same linear rate of convergence for each algorithm, irrespective of which factorization it computes. One important consequence of this randomized pivoting rule is a provable, effective bound on the numerical stability of the Jacobi eigenvalue algorithm, which addresses a longstanding open problem of Demmel and Veselić `92.

Matrix Factorizations with Uniformly Random Pivoting

TL;DR

This work unifies two foundational families of matrix factorizations—the Jacobi eigenvalue algorithm and Gram-Schmidt—with Gaussian elimination/QR through a generalized factorization framework that uses randomized pivoting. It proves that, in expectation, all special cases converge linearly at the same rate independent of the targeted factorization, and it provides a robust finite-precision stability theory for orthogonalization, Jacobi, and related decompositions. The analysis introduces a Gamma-based potential that yields explicit contraction under random size- pivoting, and extends stability results to eigenvalues and singular values via existing bounds adapted to the randomized setting. Collectively, the results offer rigorous guarantees for both the convergence and numerical stability of a broad class of factorization algorithms, with practical implications for SVD/QR computations and block-structured variants. The framework also encompasses the Kaczmarz–Kac walk and supports block, adaptive, fixed, and randomized pivoting schemes, highlighting broad applicability across numerical linear algebra.

Abstract

This paper highlights a formal connection between two families of widely used matrix factorization algorithms in numerical linear algebra. One family consists of the Jacobi eigenvalue algorithm and its variants for computing the Hermitian eigendecomposition and singular value decomposition. The other consists of Gaussian elimination and the Gram-Schmidt procedure with various pivoting rules for computing the Cholesky decomposition and QR decomposition respectively. Both families are cast as special cases of a more general class of factorization algorithms. We provide a randomized pivoting rule that applies to this general class (which differs substantially from the usual pivoting rules for Gaussian elimination / Gram-Schmidt) which admits a unified analysis of the entire class of algorithms. The result is the same linear rate of convergence for each algorithm, irrespective of which factorization it computes. One important consequence of this randomized pivoting rule is a provable, effective bound on the numerical stability of the Jacobi eigenvalue algorithm, which addresses a longstanding open problem of Demmel and Veselić `92.
Paper Structure (15 sections, 18 theorems, 118 equations, 1 table, 1 algorithm)

This paper contains 15 sections, 18 theorems, 118 equations, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Paper overview
  3. Connecting SVD and QR
  4. The core connection
  5. Pivoting rules
  6. Block variants
  7. A generalized factorization
  8. Exact arithmetic analysis
  9. Standard Jacobi analysis
  10. Our analysis
  11. Finite arithmetic analysis
  12. Control on $\Gamma(\widetilde{B}^{(t)})$
  13. Orthogonalization is stable
  14. Diagonalization and SVD are stable
  15. Characterization of Gamma

Key Result

Corollary 4.1

Let $0<\delta < 1$. Given positive definite $B\in{\mathbb{C}}^{n\times n}$, run Algorithm 1 with randomized size $k$ pivoting for iterations. Let $T_{\textnormal{acc}}=T_{\textnormal{acc}}^{(t)}$, $D=B^{(t)}$ be the values returned by the algorithm. Let $D'=D\odot I$ be just the diagonal entries of $D$. Then with probability at least $1-1/n$. Additionally, $B=T_{\textnormal{acc}}^* DT_{\textnorm

Theorems & Definitions (42)

  • Remark 1: Using recursion to compute $S$
  • Remark 2: Parallelizability
  • Remark 3: Fixed pivot size
  • Corollary 4.1
  • Lemma 4.2: Deterministic update formula
  • proof
  • Lemma 4.3: Expected update
  • proof
  • Theorem 4.4: Global linear convergence of Algorithm 1
  • proof
  • ...and 32 more