Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Superlinear Convergence of GMRES for clustered eigenvalues and its application to least squares problems

Zeyu Liao, Ken Hayami

TL;DR

This work analyzes GMRES convergence under clustered eigenvalues by deriving a Vandermonde-based upper bound on the residual that depends on the eigenstructure, the right-hand side projection, and the eigenvectors. It shows that, for diagonalizable A with clustered spectrum, the residual can decay superlinearly as the bound involving a carefully constructed polynomial vanishes on early eigenvalues; the authors apply this theory to inner-iteration preconditioning in BA-GMRES for least-squares problems using NR-SOR and demonstrate that the spectrum is steered toward 1 while eigenvectors remain well-conditioned, yielding rapid convergence. The results illuminate how eigenvalue distribution and non-normality influence GMRES, justify NR-SOR inner iterations as an effective implicit preconditioner, and motivate extending the analysis to non-diagonalizable cases via Jordan theory. Overall, the paper provides a practical framework for understanding and predicting superlinear GMRES behavior in LS contexts and guides preconditioner design to exploit eigenvalue clustering.

Abstract

The objective of this paper is to understand the superlinear convergence behavior of the GMRES method when the coefficient matrix has clustered eigenvalues. In order to understand the phenomenon, we analyze the convergence using the Vandermonde matrix which is defined using the eigenvalues of the coefficient matrix. Although eigenvalues alone cannot explain the convergence, they may provide an upper bound of the residual, together with the right hand side vector and the eigenvectors of the coefficient matrix. We show that when the coefficient matrix is diagonalizable, if the eigenvalues of the coefficient matrix are clustered, the upper bound of the convergence curve shows superlinear convergence, when the norm of the matrix obtained by decomposing the right hand side vector into the eigenvector components is not so large. We apply the analysis to explain the convergence of inner-iteration preconditioned GMRES for least squares problems.

Superlinear Convergence of GMRES for clustered eigenvalues and its application to least squares problems

TL;DR

This work analyzes GMRES convergence under clustered eigenvalues by deriving a Vandermonde-based upper bound on the residual that depends on the eigenstructure, the right-hand side projection, and the eigenvectors. It shows that, for diagonalizable A with clustered spectrum, the residual can decay superlinearly as the bound involving a carefully constructed polynomial vanishes on early eigenvalues; the authors apply this theory to inner-iteration preconditioning in BA-GMRES for least-squares problems using NR-SOR and demonstrate that the spectrum is steered toward 1 while eigenvectors remain well-conditioned, yielding rapid convergence. The results illuminate how eigenvalue distribution and non-normality influence GMRES, justify NR-SOR inner iterations as an effective implicit preconditioner, and motivate extending the analysis to non-diagonalizable cases via Jordan theory. Overall, the paper provides a practical framework for understanding and predicting superlinear GMRES behavior in LS contexts and guides preconditioner design to exploit eigenvalue clustering.

Abstract

The objective of this paper is to understand the superlinear convergence behavior of the GMRES method when the coefficient matrix has clustered eigenvalues. In order to understand the phenomenon, we analyze the convergence using the Vandermonde matrix which is defined using the eigenvalues of the coefficient matrix. Although eigenvalues alone cannot explain the convergence, they may provide an upper bound of the residual, together with the right hand side vector and the eigenvectors of the coefficient matrix. We show that when the coefficient matrix is diagonalizable, if the eigenvalues of the coefficient matrix are clustered, the upper bound of the convergence curve shows superlinear convergence, when the norm of the matrix obtained by decomposing the right hand side vector into the eigenvector components is not so large. We apply the analysis to explain the convergence of inner-iteration preconditioned GMRES for least squares problems.
Paper Structure (9 sections, 6 theorems, 90 equations, 9 figures, 3 tables, 3 algorithms)

This paper contains 9 sections, 6 theorems, 90 equations, 9 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Upper bound of the residual
  3. Clustered case
  4. Inner-iteration preconditioning
  5. Inner-iteration by NR-SOR for BA-GMRES
  6. Eigenvalue distribution for inner-iteration preconditioning
  7. Estimation for the test matrix
  8. INFLUENCE OF NON-NORMALITY
  9. Conclusions

Key Result

Theorem 1

Let $A\in\mathbb{R}^{n\times n}$ be diagonalizable ($A$ could be singular), $b\in \mathcal{R}(A)$, where $\mathcal{R}(A)$ is the range space of $A$, and $d$ is the grade of $\mathcal{K}_{k}$($A, \boldsymbol{r_0}$). Let $\boldsymbol{r_0}=c_1\boldsymbol{v_1}+c_2\boldsymbol{v_2}+\dots+c_d\boldsymbol{v_ where $V_d=[\boldsymbol{v_1}, \boldsymbol{v_2}, \dots, \boldsymbol{v_d}]$, $\boldsymbol{x_k}=\bolds

Figures (9)

  • Figure 1: The nonzero singular values of the test matrix $A$.
  • Figure 2: The nonzero eigenvalues of the normal equation matrix $A^\top A$ of the test matrix $A$.
  • Figure 3: The nonzero eigenvalues of $H=M^{-1}N$ of the test matrix $A$.
  • Figure 4: The nonzero eigenvalues of $B^{(l)}A=\rm I\it-H^l(l=\rm4)$ of the test matrix $A$.
  • Figure 5: The nonzero eigenvalues of $B^{(l)}A=\rm I\it-H^l(l=\rm8)$ of the test matrix $A$.
  • ...and 4 more figures

Theorems & Definitions (12)

  • Theorem 1
  • proof
  • Lemma 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Lemma 2
  • proof
  • ...and 2 more