Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distribution of the number of pivots needed using Gaussian elimination with partial pivoting on random matrices

John Peca-Medlin

TL;DR

This work characterizes the distribution of the number of GEPP pivot movements when solving $A x = b$ with Gaussian elimination and partial pivoting on various random matrix ensembles. It proves exact pivot-count distributions for two key models: (i) iid continuous-columns matrices yield a uniform GEPP permutation and pivot count $\Pi(A) \sim n - \Upsilon_n$, and (ii) Haar-butterfly matrices yield a uniform distribution over butterfly permutations with $\Pi(B) \sim \frac{N}{2}$ Bernoulli$(1 - \frac{1}{N})$. It further develops the theory of GEPP permutations via explicit permutation decompositions, connects these to Stirling numbers of the first kind, and demonstrates a Cantor-like pivot-location structure for butterfly ensembles. The paper introduces fixed-pivot and fixed-sparsity ensembles, analyzes their spectral properties, and conjectures a universal limiting spectral distribution $\nu_\alpha$ on the unit disk that interpolates between the circular law and a point mass at the origin as sparsity $\alpha$ increases. Numerical experiments across diverse random transforms corroborate the theoretical distributions and reveal universality patterns, with Haar orthogonal transforms serving as a benchmark upper bound for pivot movements and Haar-butterfly transforms as a minimal perturbation in many scenarios.

Abstract

Gaussian elimination with partial pivoting (GEPP) is a widely used method to solve dense linear systems. Each GEPP step uses a row transposition pivot movement if needed to ensure the leading pivot entry is maximal in magnitude for the leading column of the remaining untriangularized subsystem. We will use theoretical and numerical approaches to study how often this pivot movement is needed. We provide full distributional descriptions for the number of pivot movements needed using GEPP using particular Haar random ensembles, as well as compare these models to other common transformations from randomized numerical linear algebra. Additionally, we introduce new random ensembles with fixed pivot movement counts and fixed sparsity, $α$. Experiments estimating the empirical spectral density (ESD) of these random ensembles leads to a new conjecture on a universality class of random matrices with fixed sparsity whose scaled ESD converges to a measure on the complex unit disk that depends on $α$ and is an interpolation of the uniform measure on the unit disk and the Dirac measure at the origin.

Distribution of the number of pivots needed using Gaussian elimination with partial pivoting on random matrices

TL;DR

This work characterizes the distribution of the number of GEPP pivot movements when solving with Gaussian elimination and partial pivoting on various random matrix ensembles. It proves exact pivot-count distributions for two key models: (i) iid continuous-columns matrices yield a uniform GEPP permutation and pivot count , and (ii) Haar-butterfly matrices yield a uniform distribution over butterfly permutations with Bernoulli. It further develops the theory of GEPP permutations via explicit permutation decompositions, connects these to Stirling numbers of the first kind, and demonstrates a Cantor-like pivot-location structure for butterfly ensembles. The paper introduces fixed-pivot and fixed-sparsity ensembles, analyzes their spectral properties, and conjectures a universal limiting spectral distribution on the unit disk that interpolates between the circular law and a point mass at the origin as sparsity increases. Numerical experiments across diverse random transforms corroborate the theoretical distributions and reveal universality patterns, with Haar orthogonal transforms serving as a benchmark upper bound for pivot movements and Haar-butterfly transforms as a minimal perturbation in many scenarios.

Abstract

Gaussian elimination with partial pivoting (GEPP) is a widely used method to solve dense linear systems. Each GEPP step uses a row transposition pivot movement if needed to ensure the leading pivot entry is maximal in magnitude for the leading column of the remaining untriangularized subsystem. We will use theoretical and numerical approaches to study how often this pivot movement is needed. We provide full distributional descriptions for the number of pivot movements needed using GEPP using particular Haar random ensembles, as well as compare these models to other common transformations from randomized numerical linear algebra. Additionally, we introduce new random ensembles with fixed pivot movement counts and fixed sparsity, . Experiments estimating the empirical spectral density (ESD) of these random ensembles leads to a new conjecture on a universality class of random matrices with fixed sparsity whose scaled ESD converges to a measure on the complex unit disk that depends on and is an interpolation of the uniform measure on the unit disk and the Dirac measure at the origin.
Paper Structure (22 sections, 15 theorems, 57 equations, 6 figures, 3 tables)

This paper contains 22 sections, 15 theorems, 57 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction and background
  2. Outline of paper
  3. Distribution of GEPP pivot movements for particular random matrices
  4. Permutations from GEPP and uniform sampling of 2
  5. Proof of (I) of \ref{['thm:main']}
  6. Butterfly permutations
  7. Proof of (II) of \ref{['thm:main']}
  8. Random ensembles 2, 2, 2 and 2
  9. Eigenvalues of 2 and 2
  10. Fixed sparsity random ensembles 2 and 2
  11. Numerical experiments
  12. Min-movement (naïve) model
  13. Discussion
  14. Min-movement (worst-case) model
  15. Discussion
  16. ...and 7 more sections

Key Result

Theorem 1

(I) If $A$ is a $n\times n$ random matrix with independent columns whose first $n-1$ columns have (absolutely) continuous iid entries, then $P \sim \operatorname{Uniform}(\mathcal{P}_n)$ for $PA = LU$ the GEPP factorization of $A$ for $n \ge 2$ and $\Pi(A) \sim n - \Upsilon_n$. (II) If $B \sim \oper

Figures (6)

  • Figure 1: GEPP pivot movement configurations for Haar-butterfly permutations for $N=2^8$ and their associated probabilities, $p_k$, with the exact pivot movement locations indicated by blue
  • Figure 2: Computed eigenvalues (in blue) for $PL/\sqrt{n \sigma^2/2}$ where $n = 2^{14}=16,384$ and $PL \sim {\mathcal{PL}}_n^{\max}(\xi)$ for (a) $\xi \sim \operatorname{Uniform}([-1,1])$ where $\sigma^2 = \frac{1}{3}$, (b) $\xi \sim \operatorname{Uniform}(\mathbb D)$ where $\sigma^2 = \frac{1}{2}$, (c) $\xi \sim \operatorname{Rademacher}$ where $\sigma^2 = 1$, and (d) $\xi \sim N(0,1)$ where $\sigma^2 = 1$, mapped against the unit complex circle $\partial \mathbb D$ (in red)
  • Figure 3: Computed eigenvalues (in blue) for $PL/\sqrt{ n \sigma^2{(1-\alpha)} }$ where $n = 2^{14}=16,384$ and $PL \sim PL_n(\xi,\alpha)$ for $\xi \sim N(0,1)$ (where $\sigma^2=1$) and (a) $\alpha = 0$, (c) $\alpha = 1/4$, and (d) $\alpha=3/4$, along with (b) $n$ iid samples from $\operatorname{Uniform}(\mathbb D)$, mapped against the unit complex circle $\partial \mathbb D$ (in red)
  • Figure 4: Histogram of $10^4$ samples of pivot movement counts for random matrices of order $N = 2^4$ and $N = 2^8$.
  • Figure 5: Histogram of $10^4$ samples of pivot movement counts for 2-sided random transformations of order $N = 2^4$ and $N = 2^8$ worst-case model, $UA_N V^*$.
  • ...and 1 more figures

Theorems & Definitions (37)

  • Definition 1
  • Theorem 1
  • Corollary 1
  • Example 1
  • Example 2
  • Example 3
  • Remark 1
  • Corollary 2
  • Theorem 2: Subgroup algorithm,DiSh87
  • Corollary 3
  • ...and 27 more