Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence Properties of Nonlinear GMRES Applied to Linear Systems

Chen Greif, Yunhui He

TL;DR

This work analyzes the convergence behavior of Nonlinear GMRES (NGMRES) when applied to linear systems, clarifying its relationship with classical GMRES and Anderson acceleration. By deriving residual recurrences, polynomial representations, and orthogonality properties, the authors show that, under invertibility and decreasing GMRES residuals, full NGMRES coincides with GMRES, and that NGMRES(1) is equivalent to GMRES for symmetric or shifted-skew matrices. They also establish convergence bounds for NGMRES(m) in the real positive definite setting and derive a three-term recurrence for NGMRES(1), illustrating practical connections to CR and GMRES. Numerical experiments on nonsymmetric and shifted-skew systems corroborate the theory, highlighting when NGMRES matches GMRES and when differences arise, and emphasizing the role of preconditioning for robustness. Overall, the paper provides a rigorous foundation for understanding NGMRES as a residual-minimizing Krylov-like method for linear systems and guides its practical use and analysis.

Abstract

The Nonlinear GMRES (NGMRES) proposed by Washio and Oosterlee [Electron. Trans. Numer. Anal, 6(271-290), 1997] is an acceleration method for fixed point iterations. It has been demonstrated to be effective, but its convergence properties have not been extensively studied in the literature so far. In this work we aim to close some of this gap, by offering a convergence analysis for NGMRES applied to linear systems. A central part of our analysis focuses on identifying equivalences between NGMRES and the classical Krylov subspace GMRES method.

Convergence Properties of Nonlinear GMRES Applied to Linear Systems

TL;DR

This work analyzes the convergence behavior of Nonlinear GMRES (NGMRES) when applied to linear systems, clarifying its relationship with classical GMRES and Anderson acceleration. By deriving residual recurrences, polynomial representations, and orthogonality properties, the authors show that, under invertibility and decreasing GMRES residuals, full NGMRES coincides with GMRES, and that NGMRES(1) is equivalent to GMRES for symmetric or shifted-skew matrices. They also establish convergence bounds for NGMRES(m) in the real positive definite setting and derive a three-term recurrence for NGMRES(1), illustrating practical connections to CR and GMRES. Numerical experiments on nonsymmetric and shifted-skew systems corroborate the theory, highlighting when NGMRES matches GMRES and when differences arise, and emphasizing the role of preconditioning for robustness. Overall, the paper provides a rigorous foundation for understanding NGMRES as a residual-minimizing Krylov-like method for linear systems and guides its practical use and analysis.

Abstract

The Nonlinear GMRES (NGMRES) proposed by Washio and Oosterlee [Electron. Trans. Numer. Anal, 6(271-290), 1997] is an acceleration method for fixed point iterations. It has been demonstrated to be effective, but its convergence properties have not been extensively studied in the literature so far. In this work we aim to close some of this gap, by offering a convergence analysis for NGMRES applied to linear systems. A central part of our analysis focuses on identifying equivalences between NGMRES and the classical Krylov subspace GMRES method.

Paper Structure

This paper contains 9 sections, 15 theorems, 88 equations, 2 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Comparison of AA, GMRES, and NGMRES for solving linear systems
  3. Equivalence of NGMRES($m$) and GMRES
  4. Orthogonality properties
  5. NGMRES($m$) vs. GMRES
  6. Three-term recurrence for NGMRES(1)
  7. Convergence analysis
  8. Numerical experiments
  9. Concluding remarks

Key Result

Theorem 2.1

The residuals of NGMRES applied to fixed-point iteration eq:linear-q satisfy the recurrence relation

Figures (2)

  • Figure 1: Example \ref{['ex:conv-diff']}. Convergence history (residual norms) for NGMRES(1) vs GRMES. The left-hand side graph is for a nonsymmetric system. The right-hand side graph is for a shifted skew-symmetric system.
  • Figure 2: Example \ref{['ex:cir-n50']}. Convergence history (residual norms) for NGMRES(10) vs. GRMES. The left-hand side graph displays the first 50 iterations. The right-hand side graph displays the first 49 iterations (droped the initial guess).

Theorems & Definitions (34)

  • Remark 1.1
  • Remark 1.2
  • Theorem 2.1
  • Proof 1
  • Definition 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Remark 2.5
  • Theorem 2.6
  • Proof 2
  • ...and 24 more