Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the numerical stability of sketched GMRES

Liam Burke, Erin Carson, Yuxin Ma

TL;DR

The paper develops a backward stability analysis for preconditioned sketched GMRES (sGMRES), linking the attainable backward error to the conditioning of the Krylov basis and the preconditioned basis under randomized sketches.By introducing a key-dimension concept and carefully bounding errors in matrix products, least-squares solves, and solution recomputation, the authors derive conditions under which sGMRES is backward stable with high probability, and show how restarting can alleviate instability from ill-conditioned bases.A sharper bound is presented that explains small backward errors even when the basis is highly ill-conditioned, based on the norm product ||B|| ||y|| relative to ||x||, rather than the basis conditioning alone, and an adaptive restarting strategy is proposed to maintain stability in practice.Numerical experiments validate the theory across various sGMRES variants, demonstrating that adaptive restarting can yield backward-stable solutions with performance approaching standard GMRES, and illustrating practical guidance for choosing sketch size, restarting parameters, and when to restart.

Abstract

We perform a backward stability analysis of preconditioned sketched GMRES [Nakatsukasa and Tropp, SIAM J. Matrix Anal. Appl, 2024] for solving linear systems $Ax=b$, and show that the backward stability at iteration $i$ depends on the conditioning of the Krylov basis $B_{1:i}$ as long as the condition number of $A B_{1:i}$ can be bounded by $1/O(u)$, where $u$ is the unit roundoff. Under this condition, we show that sketched GMRES is backward stable as long as the condition number of $B_{1:i}$ is not too large. Under additional assumptions, we then show that the stability of a restarted implementation of sketched GMRES can be independent of the condition number of $B_{1:i}$, and restarted sketched GMRES is backward stable. We also derive sharper bounds that explain why the backward error can be small even in cases when the basis $B_{1:i}$ is very ill-conditioned, which has been observed in the literature but not yet explained theoretically. We present numerical experiments to demonstrate the conclusions of our analysis, and also show that adaptively restarting where appropriate allows us to recover backward stability in sketched GMRES.

On the numerical stability of sketched GMRES

TL;DR

The paper develops a backward stability analysis for preconditioned sketched GMRES (sGMRES), linking the attainable backward error to the conditioning of the Krylov basis and the preconditioned basis under randomized sketches.By introducing a key-dimension concept and carefully bounding errors in matrix products, least-squares solves, and solution recomputation, the authors derive conditions under which sGMRES is backward stable with high probability, and show how restarting can alleviate instability from ill-conditioned bases.A sharper bound is presented that explains small backward errors even when the basis is highly ill-conditioned, based on the norm product ||B|| ||y|| relative to ||x||, rather than the basis conditioning alone, and an adaptive restarting strategy is proposed to maintain stability in practice.Numerical experiments validate the theory across various sGMRES variants, demonstrating that adaptive restarting can yield backward-stable solutions with performance approaching standard GMRES, and illustrating practical guidance for choosing sketch size, restarting parameters, and when to restart.

Abstract

We perform a backward stability analysis of preconditioned sketched GMRES [Nakatsukasa and Tropp, SIAM J. Matrix Anal. Appl, 2024] for solving linear systems , and show that the backward stability at iteration depends on the conditioning of the Krylov basis as long as the condition number of can be bounded by , where is the unit roundoff. Under this condition, we show that sketched GMRES is backward stable as long as the condition number of is not too large. Under additional assumptions, we then show that the stability of a restarted implementation of sketched GMRES can be independent of the condition number of , and restarted sketched GMRES is backward stable. We also derive sharper bounds that explain why the backward error can be small even in cases when the basis is very ill-conditioned, which has been observed in the literature but not yet explained theoretically. We present numerical experiments to demonstrate the conclusions of our analysis, and also show that adaptively restarting where appropriate allows us to recover backward stability in sketched GMRES.

Paper Structure

This paper contains 15 sections, 4 theorems, 64 equations, 6 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preconditioned GMRES
  3. Preconditioned Sketched GMRES
  4. Backward stability of preconditioned sketched GMRES
  5. Step \ref{['eq:line-C']}: Matrix--matrix product
  6. Step \ref{['eq:line-ls']}: Least squares solver
  7. Step \ref{['eq:line-x']}: Computation of the approximate solution
  8. Adaptive restarting sGMRES
  9. Numerical Experiments
  10. Illustration of the backward error bound
  11. Attainable worst case for sGMRES
  12. Comparison between the bounds shown in Theorem \ref{['thm:stability_sgmres']} and Lemma \ref{['lem:backward-sGMRES']}
  13. Comparison between different sGMRES variants with and without restarting
  14. sGMRES with the adaptive restarting strategy
  15. Conclusions

Key Result

Lemma 1

Assume that $\hat{W}_{1:k}$ satisfies where $S$ is an $\epsilon$-subspace embedding for the space spanned by the matrix $$ satisfying eq:sketch, and $c_1(n, k)$ and $c_2(n, k)$ are low degree polynomials related to $n$ and $k$. Also assume that $\hat{y}^{(k)}$ satisfies eq:ls-S. Then where $c(n, k)$ is also a low degree polynomial related to $n$ and $k$.

Figures (6)

  • Figure 1: Example 1: From the left to right, the plots show the backward error, $\tau_{i}$ and the approximate condition numbers of basis $B_{i}$ and $A B_{i}$, monitored every iteration, for different implementations of sGMRES, and an implementation of standard GMRES.
  • Figure 2: Example 2: From the left to right, the plots show the backward error, $\tau_{i}$ and the approximate condition numbers of the $B_{i}$ and $A B_{i}$, monitored every iteration, for different implementations of sGMRES, and an implementation of standard GMRES.
  • Figure 3: Example 3: From the left to right, the plots show the backward error, $\tau_{i}$, and the approximate condition numbers of $B_{i}$ and $A B_{i}$, monitored every iteration, for different implementations of sGMRES, and an implementation of standard GMRES.
  • Figure 4: Adaptive restarting experiment. From left to right, we plot the backward error, the value of $\bar{t}$, and the value of $\tau$ for fs_760_1, where $\bar{t} = \max(t, i)$ represents the number of columns against which $B_i$ needs to perform orthogonalization in Line \ref{['line:algo_trunc:forloop']} of Algorithm \ref{['alg:trunc_Arnoldi']}.
  • Figure 5: Adaptive restarting experiment. From left to right, we plot the backward error, the value of $\bar{t}$, and the value of $\tau$ for sherman2, where $\bar{t} = \max(t, i)$ represents the number of columns against which $B_i$ needs to perform orthogonalization in Line \ref{['line:algo_trunc:forloop']} of Algorithm \ref{['alg:trunc_Arnoldi']}.
  • ...and 1 more figures

Theorems & Definitions (14)

  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Remark 1
  • Remark 2
  • Remark 3
  • Lemma 2
  • Remark 4
  • Remark 5
  • ...and 4 more