Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating a restarted Krylov method for matrix functions with randomization

Nicolas L. Guidotti, Per-Gunnar Martinsson, Juan A. Acebrón, José Monteiro

TL;DR

This work introduces a randomized acceleration for restarted Krylov methods to compute f(A)b efficiently for large sparse matrices. By replacing the Arnoldi step with a Randomized Gram-Schmidt (RGS) or related sketch-based approach, the method builds a well-conditioned, compact Krylov basis that can be restarted with modest overhead. Empirical results on large finite-element problems show that randomized restarted Krylov often outperforms classical restarted methods, sometimes accelerating convergence and providing superior spectrum representation, while maintaining accuracy and stability. The findings highlight meaningful practical gains for PDE discretizations and point to theoretical questions about why randomization improves convergence in some cases and how to bound its behavior in ill-conditioned settings.

Abstract

Many scientific applications require the evaluation of the action of the matrix function over a vector and the most common methods for this task are those based on the Krylov subspace. Since the orthogonalization cost and memory requirement can quickly become overwhelming as the basis grows, the Krylov method is often restarted after a few iterations. This paper proposes a new acceleration technique for restarted Krylov methods based on randomization. The numerical experiments show that the randomized method greatly outperforms the classical approach with the same level of accuracy. In fact, randomization can actually improve the convergence rate of restarted methods in some cases. The paper also compares the performance and stability of the randomized methods proposed so far for solving very large finite element problems, complementing the numerical analyses from previous studies.

Accelerating a restarted Krylov method for matrix functions with randomization

TL;DR

This work introduces a randomized acceleration for restarted Krylov methods to compute f(A)b efficiently for large sparse matrices. By replacing the Arnoldi step with a Randomized Gram-Schmidt (RGS) or related sketch-based approach, the method builds a well-conditioned, compact Krylov basis that can be restarted with modest overhead. Empirical results on large finite-element problems show that randomized restarted Krylov often outperforms classical restarted methods, sometimes accelerating convergence and providing superior spectrum representation, while maintaining accuracy and stability. The findings highlight meaningful practical gains for PDE discretizations and point to theoretical questions about why randomization improves convergence in some cases and how to bound its behavior in ill-conditioned settings.

Abstract

Many scientific applications require the evaluation of the action of the matrix function over a vector and the most common methods for this task are those based on the Krylov subspace. Since the orthogonalization cost and memory requirement can quickly become overwhelming as the basis grows, the Krylov method is often restarted after a few iterations. This paper proposes a new acceleration technique for restarted Krylov methods based on randomization. The numerical experiments show that the randomized method greatly outperforms the classical approach with the same level of accuracy. In fact, randomization can actually improve the convergence rate of restarted methods in some cases. The paper also compares the performance and stability of the randomized methods proposed so far for solving very large finite element problems, complementing the numerical analyses from previous studies.

Paper Structure

This paper contains 13 sections, 3 theorems, 44 equations, 9 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Theory
  3. Notation
  4. Krylov Subspace Methods
  5. Random Sketching
  6. Randomized Krylov
  7. Related Works
  8. Restarted Randomized Krylov
  9. Numerical Experiments
  10. Convection-Diffusion
  11. Circular Membrane
  12. Performance
  13. Conclusion

Key Result

Lemma 2.1

If $\mathbf{S} \in \mathbb{C}^{d \times n}$ is an oblivious subspace embedding of $\mathcal{K}_m(\mathbf{A}, \mathbf{b})$, then the singular values of $\mathbf{W}_m$ are bounded by

Figures (9)

  • Figure 1: The finite element mesh and its boundary conditions for the convection-diffusion example.
  • Figure 2: The spectrum of the matrix $\mathbf{L}$ for a coarse mesh with the size of the finite elements ranging between $0.12$ and $0.6$.
  • Figure 3: Convergence curves for different numerical methods when solving the convection-diffusion problem. Both the restart length $m_r$ and truncation parameter $k$ were set to $20$. In terms of the sketching dimension $d$, we set $d = 2400$ for rand, rand-ls and sFOM and $d = 320$ for restart-rand.
  • Figure 4: The Ritz values for the convection-diffusion problem with $\alpha = 0.1, \beta = 0.01$, i.e., for test (a). The restart length was set to $20$.
  • Figure 5: Convergence curves for restart and restart-rand for different restart lengths $m_r$ for the convection-diffusion example. The sketching dimension $d$ was set to $320$.
  • ...and 4 more figures

Theorems & Definitions (4)

  • Lemma 2.1: Corollary 2.2 from balabanov_randomized_2022
  • Lemma 2.2
  • Proof 1
  • Lemma 2.3: Theorem 2.4 from eiermann_restarted_2006