Extending Elman's Bound for GMRES

TL;DR

The paper addresses GMRES convergence when the numerical range of the system matrix touches the origin, which undermines Elman's classic bound. It introduces a Lyapunov-based method to construct a family of inner products via ${\bf A}^*{\bf G}+{\bf G}{\bf A}={\bf C}$, so that the transformed numerical range $W_{\bf G}({\bf A})$ lies in the right half-plane and Elman's bound applies to GMRES run in the Euclidean norm, up to a distortion factor $\sqrt{\kappa_2({\bf G})}$. Lyapunov inverse iteration generates a spectrum of such inner products, balancing the location of $W_{\bf G}({\bf A})$ against ${\sqrt{\kappa({\bf G})}}$, and is illustrated with applications to a damped mechanical system and a preconditioned KKT system. The framework also connects with diagonalization-based bounds and offers a path to tighter GMRES estimates via bounds on $W_{\bf G}({\bf A})$ or Crouzeix–Palencia-type results, highlighting inner-product design as a potential lever for nonnormal GMRES convergence. Overall, this work provides a theoretical tool for analyzing GMRES and hints at practical inner-product–based preconditioners to enhance convergence in challenging, nonnormal settings.

Abstract

If the numerical range of a matrix is contained in the right half of the complex plane, the GMRES algorithm for solving linear systems will reduce the norm of the residual at every iteration. In his Ph.D. dissertation, Howard Elman derived a bound that guarantees convergence. When the numerical range contains the origin, GMRES need not make progress at every step and Elman's bound does not apply, even if all the eigenvalues are located in the right half-plane. However by solving a Lyapunov equation, one can construct an inner product in which the numerical range is contained in the right half-plane. One can then bound GMRES (run in the standard Euclidean norm) by applying Elman's bound in this new inner product, at the cost of a multiplicative constant that characterizes the distortion caused by the change of inner product. Using Lyapunov inverse iteration, one can build a family of such inner products, trading off the location of the numerical range with the size of constant. This approach complements techniques that Greenbaum and colleagues have recently proposed for excising the origin from the numerical range to gain greater insight into the convergence of GMRES for nonnormal matrices.

Figures (8)

• Figure 1: Illustration of the bound \ref{['eq:nrbound']}, in which the numerical range $W({\bf A})$ is contained in a disk segment: the left edge of the disk segment (blue) is determined by $\mu({\bf A})$, the leftmost eigenvalue of the Hermitian part of ${\bf A}$; this bound is sharp, i.e., there exists $z\in W({\bf A})$ with ${\rm Re}(z)=\mu({\bf A})$. The circular arc (red) is determined by $\|{\bf A}\|_2$; it need not be sharp, but cannot be off by more than a factor of two: there exists $z\in W({\bf A})$ such that $|z|\ge \frac{1}{2}\|{\bf A}\|_2$HJ13.
• Figure 2: An optimally damped mechanical system \ref{['eq:waveA']} for which the Lyapunov equation \ref{['eq:lyap_cox']} has a tidy solution, adapted from Cox98a. For this example ($N=64$, $n=128$), the origin $(+)$ is embedded deep in the numerical range $W({\bf A})$. The inner product defined by ${\bf G}$ in \ref{['eq:waveG']} gives a much smaller numerical range $W_{\bf G}({\bf A})$ contained in the left half-plane, with ${\sqrt{\kappa_2({\bf G})}} = 225.035$. The eigenvalues of ${\bf A}$ (which appear as the black vertical segment in the right plot) include a $2\times 2$ Jordan block. (The horizontal and vertical axes are scaled differently in the right plot.)
• Figure 3: A preconditioned KKT example, adapted from FRSW98. Here ${\bf B}\in\mathbb{R}^{64\times 128}$ has normally distributed random entries and $\eta = 2@\|{\bf B}\|_2+0.1$. On the left, the numerical range is a large region touching the origin: $W({\bf A})\cap\mathbb{R} = [0,\eta]$. On the right, ${\bf W}_{\bf G}({\bf A})$ is simply the convex hull of the spectrum, $W_{\bf G}({\bf A}) \approx [0.35633,\eta]$, with ${\sqrt{\kappa_2({\bf G})}} \approx 27.6528$.
• Figure 4: Jordan block example. Solving ${\bf A}^*{\bf G}+{\bf G}{\bf A} = {\bf I}$ yields an inner product in which $W_{\bf G}({\bf A})$ (blue region) is in the right half-plane, but barely: $\mu_{\bf G}({\bf A})\approx 3.6\times 10^{-9}$, with ${\sqrt{\kappa({\bf G})}} \approx 2.4\times 10^4$. These quantities are consistent with the slow convergence of GMRES observed on the right (for 100 random ${\bf b}$ vectors).
• Figure 5: Integration matrix example. Solving ${\bf A}^*{\bf G}+{\bf G}{\bf A} = {\bf I}$ yields an inner product in which $W_{\bf G}({\bf A})$ (blue region) is well separated from the origin: $\mu_{\bf G}({\bf A})\approx 0.17$, with ${\sqrt{\kappa({\bf G})}} \approx 3.5$. These quantities are consistent with the rapid convergence of GMRES observed on the right (for 100 random ${\bf b}$ vectors).

Theorems & Definitions (4)

• Theorem 1
• Corollary 1
• Corollary 2
• Corollary 3