Convergence Analysis of An Alternating Nonlinear GMRES on Linear Systems

TL;DR

This work analyzes the convergence of a one-step NGMRES applied to linear systems in fixed-point form $u_{k+1}=Mu_k+b$ with $M=I-A$, and introduces an alternating variant aNGMRES(m,p) that applies NGMRES every $p$ iterations to reduce cost. It provides quantitative residual bounds in two key matrix regimes: diagonalizable $M$ and SPD $M$, linking gains to Chebyshev-type polynomials and Krylov subspace concepts. The authors establish precise relationships between aNGMRES and GMRES, including that aNGMRES($m,m+1$) matches GMRES($m+1$) at period ends and that aNGMRES($∞,p$) aligns with full GMRES at interval endpoints under suitable conditions. Convergence results show that, for contractive fixed-point iterations, aNGMRES converges and its rate depends on the eigenstructure and chosen $(m,p)$, with numerical experiments validating theory and illustrating robustness to stagnation and applicability to nonlinear problems.

Abstract

In this work, we develop an alternating nonlinear Generalized Minimum Residual (NGMRES) algorithm with depth $m$ and periodicity $p$, denoted by aNGMRES($m, p$), applied to linear systems. We provide a theoretical analysis to quantify by how much one-step NGMRES($m$) using Richardson iterations as initial guesses can improve the convergence speed of the underlying fixed-point iteration for diagonalizable and symmetric positive definite cases. Our theoretical analysis gives us a better understanding of which factors affect the convergence speed. Moreover, under certain conditions, we prove the periodic equivalence between the proposed aNGMRES applied to Richardson iteration and GMRES. Specifically, aNGMRES($\infty,p$) and full GMRES are identical at the iteration index $jp$. Therefore, aNGMRES($\infty,p$) can be regarded as an alternative to GMRES for solving linear systems. For finite $m$, the iterates of aNGMRES($m,m+1$) and restarted GMRES (GMRES($m+1$)) are the same at the end of each periodic interval of length $p$, i.e, at the iteration index $jp$. In Addition, we present a convergence analysis of aNGMRES when applied to accelerate Richardson iteration. The advantages of aNGMRES($m,p$) method are that there is no need to solve a least-squares problem at each iteration which can reduce the computational cost, and it can enhance the robustness against stagnations, which could occur for NGMRES($m$).

Figures (10)

• Figure 1: Residual norm convergence history of aNGMRES(3,4) vs. GMRES(4) for Example \ref{['ex:cirex']}.
• Figure 2: Residual norm convergence history for Example \ref{['ex:cirex']}. Left: aNGMRES($\infty$,4) vs. GMRES. Right: aNGMRES($\infty$,5) vs. GMRES.
• Figure 3: Residual norm convergence history for Example \ref{['ex:blockCirex']}: aNGMES(2,3) vs. GMRES(3).
• Figure 4: Residual norm convergence history for Example \ref{['ex:blockCirex']}: aNGMRES($\infty$,$p$) with $p=1, 2, 3,4$ vs. GMRES.
• Figure 5: Residual norm convergence history for Example \ref{['ex:Lap']}: aNGMRES($\infty$,$3$) vs. GMRES.

Theorems & Definitions (29)

• Remark 2.1
• Lemma 2.1
• Lemma 2.2
• Lemma 2.3
• proof
• Theorem 2.1
• proof
• Theorem 2.2
• proof
• Lemma 2.4