Growth factors of orthogonal matrices and local behavior of Gaussian elimination with partial and complete pivoting

TL;DR

An empirical study on a family of exponential GEPP growth matrices whose polynomial behavior in small neighborhoods limits to the initial GECP growth factor is presented, which will lead to new empirical lower bounds on how much worse G ECP can behave compared to GEPp in terms of growth.

Abstract

Gaussian elimination (GE) is the most used dense linear solver. Error analysis of GE with selected pivoting strategies on well-conditioned systems can focus on studying the behavior of growth factors. Although exponential growth is possible with GE with partial pivoting (GEPP), growth tends to stay much smaller in practice. Support for this behavior was provided recently by Huang and Tikhomirov's average-case analysis of GEPP, which showed GEPP growth factors for Gaussian matrices stay at most polynomial with very high probability. GE with complete pivoting (GECP) has also seen a lot of recent interest, with improvements to both lower and upper bounds on worst-case GECP growth provided by Bisain, Edelman and Urschel in 2023. We are interested in studying how GEPP and GECP behave on the same linear systems as well as studying large growth on particular subclasses of matrices, including orthogonal matrices. Moreover, as a means to better address the question of why large growth is rarely encountered, we further study matrices with a large difference in growth between using GEPP and GECP, and we explore how the smaller growth strategy dominates behavior in a small neighborhood of the initial matrix.

Figures (8)

• Figure 1: Normalized histogram of $512\times 512$ grid for each pair of $\rho^{\operatorname{GEPP}}$ and $\rho^{\operatorname{GECP}}$ using $10^6$$\operatorname{Haar}(\operatorname{O}(3))$ samples, with marks at $(2.25,2.25)$ and $(3,\sqrt3)$ corresponding to $b_3$ and $d_3$.
• Figure 1: Plot of the computed lower bounds for $c_n$.
• Figure 1: Normalized histogram of each pair of $\rho^{\operatorname{GEPP}}$ and $\rho^{\operatorname{GECP}}$ using $10^6$ samples of (a) $Q(\boldsymbol \theta)B_3$ for $\boldsymbol \theta \sim \operatorname{Uniform}(B_r({\mathbf 0})) \subset \mathbb R^{3}, r = \varepsilon/\sqrt{6}$, and (b) $B_3 + (\varepsilon/\sqrt 3) G$ for $G$ for $G \sim \operatorname{Ginibre}(3,3)$, each using $\varepsilon = 10^{-3}$.
• Figure 2: Plot of the average computed growth factors for $10^6$ samples of $\operatorname{Haar} \operatorname{O}(n)$ and $\operatorname{Ginibre}(n,n)$ for $n=2:20,50,100$.
• Figure 2: Normalized histogram of each pair of $\rho^{\operatorname{GEPP}}$ and $\rho^{\operatorname{GECP}}$ using $10^6$ samples of (a) $UQ_4$ for $U = Q(\boldsymbol{\theta})$, $\boldsymbol \theta \sim \operatorname{Uniform}(B_r({\mathbf 0})) \subset \mathbb R^{6}, r = \varepsilon/\sqrt{12}$, and (b) $Q_4 + (\varepsilon/2) G$ for $G$ for $G \sim \operatorname{Ginibre}(4,4)$, each using $\varepsilon = 10^{-3}$. The line $\rho^{\operatorname{GECP}} = \sqrt{11}/2$ is included for comparison.

Theorems & Definitions (22)

• Proposition 2.1
• Proposition 3.1
• Example 3.2
• Corollary 3.3
• Lemma 3.4
• Proof 1
• Remark 3.5
• Theorem 3.6: HiHi89
• Corollary 3.7
• Remark 3.8