Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Average-case analysis of the Gaussian Elimination with Partial Pivoting

Han Huang, Konstantin Tikhomirov

TL;DR

A (partial) theoretical justification of this phenomenon is obtained by showing that, given the random Gaussian coefficient matrix, the growth factor of the Gaussian Elimination with Partial Pivoting is at most polynomially large in $n$ with probability close to one.

Abstract

The Gaussian Elimination with Partial Pivoting (GEPP) is a classical algorithm for solving systems of linear equations. Although in specific cases the loss of precision in GEPP due to roundoff errors can be very significant, empirical evidence strongly suggests that for a {\it typical} square coefficient matrix, GEPP is numerically stable. We obtain a (partial) theoretical justification of this phenomenon by showing that, given the random $n\times n$ standard Gaussian coefficient matrix $A$, the {\it growth factor} of the Gaussian Elimination with Partial Pivoting is at most polynomially large in $n$ with probability close to one. This implies that with probability close to one the number of bits of precision sufficient to solve $Ax = b$ to $m$ bits of accuracy using GEPP is $m+O(\log n)$, which improves an earlier estimate $m+O(\log^2 n)$ of Sankar, and which we conjecture to be optimal by the order of magnitude. We further provide tail estimates of the growth factor which can be used to support the empirical observation that GEPP is more stable than the Gaussian Elimination with no pivoting.

Average-case analysis of the Gaussian Elimination with Partial Pivoting

TL;DR

A (partial) theoretical justification of this phenomenon is obtained by showing that, given the random Gaussian coefficient matrix, the growth factor of the Gaussian Elimination with Partial Pivoting is at most polynomially large in with probability close to one.

Abstract

The Gaussian Elimination with Partial Pivoting (GEPP) is a classical algorithm for solving systems of linear equations. Although in specific cases the loss of precision in GEPP due to roundoff errors can be very significant, empirical evidence strongly suggests that for a {\it typical} square coefficient matrix, GEPP is numerically stable. We obtain a (partial) theoretical justification of this phenomenon by showing that, given the random standard Gaussian coefficient matrix , the {\it growth factor} of the Gaussian Elimination with Partial Pivoting is at most polynomially large in with probability close to one. This implies that with probability close to one the number of bits of precision sufficient to solve to bits of accuracy using GEPP is , which improves an earlier estimate of Sankar, and which we conjecture to be optimal by the order of magnitude. We further provide tail estimates of the growth factor which can be used to support the empirical observation that GEPP is more stable than the Gaussian Elimination with no pivoting.
Paper Structure (17 sections, 33 theorems, 352 equations, 2 algorithms)

This paper contains 17 sections, 33 theorems, 352 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Outline of the proof
  3. Sankar's argument
  4. High-level structure of the proof of the main theorem
  5. Intermediate singular values of partially random block matrices
  6. Random polytopes, and distances to pivot rows
  7. A recursive argument
  8. Choice of parameters and the growth function
  9. Good events, and probability estimates
  10. The smallest singular value and the growth factor in exact arithmetic
  11. Distance to subspaces
  12. The smallest singular value of $A_{I_r,[r]}$
  13. Estimating the growth factor in exact arithmetic
  14. GEPP in floating point arithmetic
  15. Further questions
  16. ...and 2 more sections

Key Result

Theorem A

There are universal constants $C,\tilde{C}>1$ and a function $\tilde{n}:[1,\infty)\to\mathbb{N}$ with the following property. Let $p\geq 1$, and let $n\geq \tilde{n}(p)$. Then with probability at least $1-{\bf u}^{1/8}\,n^{\tilde{C}}$, the GEPP for ${\rm fl}(A)$ succeeds in floating point arithmetic and the computed permutation matrix $\hat{P}$ agrees with the matrix $P$ from the $PLU$--factoriza

Theorems & Definitions (69)

  • Theorem A
  • Theorem 3.1
  • Proposition 3.2: Singular values of random matrices with continuous distributions
  • Proposition 3.3
  • Lemma 3.4
  • proof
  • Lemma 3.5
  • proof
  • Lemma 4.1
  • proof
  • ...and 59 more