Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GMRES with randomized sketching and deflated restarting

Liam Burke, Stefan Güttel, Kirk M. Soodhalter

TL;DR

GMRES-SDR addresses efficiently solving sequences of linear systems by combining randomized sketching with deflated restarting in a recycling framework.It augments an unprojected Krylov subspace with a sketched projection subproblem, updates the augmentation space via sketched harmonic Ritz vectors, and controls residuals using sketch-based bounds.Theoretical contributions show GMRES-SDR is a projection method under a semi-inner product induced by the sketch, and relate sketched GMRES to sketched FOM through MR/OR structure and Arnoldi relations.Numerical experiments demonstrate that GMRES-SDR reduces inner products and, in many cases, total runtime relative to GCRO-DR and GMRES-DR while maintaining or improving convergence on large-scale problems.

Abstract

We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching and deflated restarting in a way that avoids orthogononalizing a full Krylov basis. We call this new method GMRES-SDR (sketched deflated restarting). With this new method, we provide new theory, which initially characterizes unaugmented sketched GMRES as a projection method for which the projectors involve the sketching operator. We demonstrate that sketched GMRES and its sibling method sketched FOM are an MR/OR pairing, just like GMRES and FOM. We furthermore obtain residual convergence estimates. Building on this, we characterize GMRES-SDR also in terms of sketching-based projectors. Compression of the augmented Krylov subspace for recycling is performed using a sketched version of harmonic Ritz vectors. We present results of numerical experiments demonstrating the effectiveness of GMRES-SDR over competitor methods such as GMRES-DR and GCRO-DR.

GMRES with randomized sketching and deflated restarting

TL;DR

GMRES-SDR addresses efficiently solving sequences of linear systems by combining randomized sketching with deflated restarting in a recycling framework.It augments an unprojected Krylov subspace with a sketched projection subproblem, updates the augmentation space via sketched harmonic Ritz vectors, and controls residuals using sketch-based bounds.Theoretical contributions show GMRES-SDR is a projection method under a semi-inner product induced by the sketch, and relate sketched GMRES to sketched FOM through MR/OR structure and Arnoldi relations.Numerical experiments demonstrate that GMRES-SDR reduces inner products and, in many cases, total runtime relative to GCRO-DR and GMRES-DR while maintaining or improving convergence on large-scale problems.

Abstract

We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching and deflated restarting in a way that avoids orthogononalizing a full Krylov basis. We call this new method GMRES-SDR (sketched deflated restarting). With this new method, we provide new theory, which initially characterizes unaugmented sketched GMRES as a projection method for which the projectors involve the sketching operator. We demonstrate that sketched GMRES and its sibling method sketched FOM are an MR/OR pairing, just like GMRES and FOM. We furthermore obtain residual convergence estimates. Building on this, we characterize GMRES-SDR also in terms of sketching-based projectors. Compression of the augmented Krylov subspace for recycling is performed using a sketched version of harmonic Ritz vectors. We present results of numerical experiments demonstrating the effectiveness of GMRES-SDR over competitor methods such as GMRES-DR and GCRO-DR.
Paper Structure (25 sections, 10 theorems, 83 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 25 sections, 10 theorems, 83 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Relevant background
  3. The classic GMRES method
  4. Recycling and augmentation of minimum residual methods
  5. GMRES-DR and GCRO-DR are sometimes equivalent
  6. Derivation of GMRES-SDR
  7. Implementation
  8. Krylov basis generation
  9. Solution of the projected least squares problem
  10. Residual control and updating safety
  11. Harmonic updating of the augmentation space
  12. Solving slowly varying linear systems
  13. Analysis of sketched GMRES
  14. Sketching the constraint space for one cycle of sketched GMRES
  15. Sketched GMRES is a projection method
  16. ...and 10 more sections

Key Result

Lemma 4.1

The matrices $\widehat{\Pi}_{\mathcal{K}_m}$ and $\widehat{\Phi}_{\mathcal{K}_m}$ are both projectors, with $\mathrm{Range}\left( \widehat{\Phi}_{\mathcal{K}_m} \right)\subseteq\mathrm{Range}( A V_m)$ with null space containing $\mathrm{Range}( S^\ast S A V_m)^\perp$ and $\mathrm{Range}\left( \wideh

Figures (6)

  • Figure 1: Example showing measured sketching distortion factors as the GMRES-SDR iteration progresses, as well as the ratio between the true and sketched residuals. In this example, the Krylov dimension of size $m = 100$, a truncation parameter of $t=2$, a recycling subspace dimension of size $k=20$, and a sketching parameter $s = 500$ was used.
  • Figure 1: Contours of $d(\varepsilon, \theta)/\sin\theta$, the difference between the estimated sketched residual norm in the first bound from \ref{['eq:sketched-residual-bound-estimate']} and the true residual norm (for $\varepsilon=0)$, (relative to the size of $\sin\theta$); i.e., $d(\varepsilon, \theta) = \sqrt{ \dfrac{ \sin^2\theta +2\varepsilon(1 + \cos\theta) }{ 1-\varepsilon^2 } } - \sin\theta$. Contours are generated for $d(\varepsilon, \theta)/\sin\theta\in\left\lbrace 1/16, 1/8, 1/4, 1/2, 1\right\rbrace$, which should be understood as percentage relative difference.
  • Figure 1: Testing the Givens rotation-based residual estimator $\left\|\tilde{\mathbf r}_m\right\|~\leq~\sqrt{\dfrac{s_m^2 + 2\varepsilon(1+c_m)}{1-\varepsilon^2}}\left\|\tilde{\mathbf r}_{m-1}\right\|$ for a problem of Neumann type from the experiments in \ref{['section.Neumann-experiments']} for various proposed values of $\varepsilon$.
  • Figure 1: Convergence curves obtained from solving a single linear system $A\mathbf x = \mathbf b$ (left), and a sequence of $5$ linear systems (right) with the vas_stokes_1M matrix, to a residual tolerance of $10^{-6}$. All non-augmented methods use a maximum number of $m = 100$ Arnoldi iterations, while the augmented methods use an augmentation subspace of dimension $k = 20$, and take a maximum of $m-k$ iterations. All sketching methods take $s = 10(m+k)$ and an Arnoldi truncation parameter $t = 2$. A maximum of $10$ restarts is allowed for all methods.
  • Figure 2: Convergence curves for the first (left) and last (right) Neumann problem.
  • ...and 1 more figures

Theorems & Definitions (21)

  • Lemma 4.1
  • Proof 1
  • Lemma 4.2
  • Proof 2
  • Lemma 4.3
  • Proof 3
  • Corollary 4.4
  • Proof 4
  • Remark 4.5
  • Theorem 4.6
  • ...and 11 more