Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Near-optimal convergence of the full orthogonalization method

Tyler Chen, Gérard Meurant

TL;DR

The paper examines Full Orthogonalization Method (FOM) and GMRES for solving non-symmetric linear systems $\mathbf{A}\mathbf{x}=\mathbf{b}$, noting FOM's oscillatory residuals versus GMRES's residual-optimal behavior. It proves a near-optimality bound: for every iteration $k\ge 1$, $\min_{0\le j\le k} \|\mathbf{r}_j^{\mathrm{F}}\|_2 \le \sqrt{k+1}\, \|\mathbf{r}_k^{\mathrm{G}}\|_2$, linking FOM’s cumulative convergence to the current GMRES residual via the scalar sequence $\{\vartheta_j\}$. The bound is sharp, demonstrated by constructing matrices for which equality holds, establishing that the factor $\sqrt{k+1}$ cannot be improved. The results extend to matrix-function computations through Arnoldi-FA, providing practical a posteriori error estimates and offering theoretical justification for using FOM and Arnoldi-based methods in computing $f(\mathbf{A})\mathbf{b}$.

Abstract

We establish a near-optimality guarantee for the full orthogonalization method (FOM), showing that the overall convergence of FOM is nearly as good as GMRES. In particular, we prove that at every iteration $k$, there exists an iteration $j\leq k$ for which the FOM residual norm at iteration $j$ is no more than $\sqrt{k+1}$ times larger than the GMRES residual norm at iteration $k$. This bound is sharp, and it has implications for algorithms for approximating the action of a matrix function on a vector.\end{abstract}

Near-optimal convergence of the full orthogonalization method

TL;DR

The paper examines Full Orthogonalization Method (FOM) and GMRES for solving non-symmetric linear systems , noting FOM's oscillatory residuals versus GMRES's residual-optimal behavior. It proves a near-optimality bound: for every iteration , , linking FOM’s cumulative convergence to the current GMRES residual via the scalar sequence . The bound is sharp, demonstrated by constructing matrices for which equality holds, establishing that the factor cannot be improved. The results extend to matrix-function computations through Arnoldi-FA, providing practical a posteriori error estimates and offering theoretical justification for using FOM and Arnoldi-based methods in computing .

Abstract

We establish a near-optimality guarantee for the full orthogonalization method (FOM), showing that the overall convergence of FOM is nearly as good as GMRES. In particular, we prove that at every iteration , there exists an iteration for which the FOM residual norm at iteration is no more than times larger than the GMRES residual norm at iteration . This bound is sharp, and it has implications for algorithms for approximating the action of a matrix function on a vector.\end{abstract}
Paper Structure (7 sections, 3 theorems, 18 equations, 3 figures)

This paper contains 7 sections, 3 theorems, 18 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Convergence bounds
  3. Discussion
  4. Past work
  5. Error norm
  6. Connection to matrix functions
  7. Conclusion

Key Result

Theorem 2.1

For every $k\geq 1$,

Figures (3)

  • Figure 1.1: Residual norms for FOM $\|\mathbf{r}_k^{\mathrm{F}}\|_2$ (solid) and GMRES $\|\mathbf{r}_k^{\mathrm{G}}\|_2$ (dash-dot) for a symmetric matrix with eigenvalues in $[-10,-1]\cup[1,20]$ and the steam2 matrix. All curves are normalized by $\|\mathbf{b}\|_2$. While the residual norms for FOM jump up and down, they exhibit a generally downward trend which mirrors the convergence of the GMRES residual norms.
  • Figure 2.1: Best FOM residual so far $\min_{j\leq k} \|\mathbf{r}_j^{\mathrm{F}}\|_2$ (solid), bound of \ref{['thm:main']} (dashed), and residual norms for FOM $\|\mathbf{r}_k^{\mathrm{F}}\|_2$ (solid grey) and GMRES $\|\mathbf{r}_k^{\mathrm{G}}\|_2$ (dash-dot). All curves are normalized by $\|\mathbf{b}\|_2$. While FOM exhibits oscillatory convergence, \ref{['thm:main']} ensures that the best FOM residual norm seen so far matches closely with the GMRES residual norm.
  • Figure 3.1: Best FOM error so far $\min_{j\leq k} \|\mathbf{A}^{-1}\mathbf{b} - \mathbf{x}_k^{\mathrm{F}}\|_2$ (solid), error for FOM $\|\mathbf{A}^{-1}\mathbf{b} - \mathbf{x}_k^{\mathrm{F}}\|_2$ (solid grey) and GMRES $\|\mathbf{A}^{-1}\mathbf{b} - \mathbf{x}_k^{\mathrm{G}}\|_2$ (dash-dot). All curves are normalized by $\|\mathbf{A}^{-1}\mathbf{b}\|_2$. Note that the FOM convergence is slightly better than GMRES, indicating FOM may be preferable to GMRES in some situations.

Theorems & Definitions (4)

  • Theorem 2.1
  • Theorem 2.2: Theorem 3.12 in meurant_tebbens_20
  • Theorem 2.3
  • proof