Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Restoring similarity in randomized Krylov methods with applications to eigenvalue problems and matrix functions

Laura Grigori, Daniel Kressner, Nian Shao, Igor Simunec

TL;DR

This work addresses the lack of similarity between Hessenberg matrices produced by standard and randomized Arnoldi methods, which can disrupt convergence in eigenvalue computations and matrix-function evaluations. It introduces a similarity-restoring correction (RAC) that enforces orthogonality of the last Krylov vector via a least-squares step, yielding a decomposition $AU_m=U_m\widehat{H}_m+\widehat{u}_{m+1}c_m^\mathsf{H}$ with $U_m^\mathsf{H}\widehat{u}_{m+1}=0$, ensuring $\widehat{H}_m$ is similar to the standard $\widehat{G}_m$. The method is applied to a similarity-restoring randomized Krylov–Schur algorithm (SRR-KS) and its matrix-function counterpart, proving exact-equivalence to standard KS in exact arithmetic and establishing backward stability. Numerical experiments show that SRR-KS is as fast as randomized Krylov–Schur and as robust as the standard approach, with significant mitigation of convergence spikes in both eigenvalue problems and $f(A)b$ computations. Overall, the paper delivers a practical, stable, and scalable framework for reliable large-scale Krylov-based computations.

Abstract

The randomized Arnoldi process has been used in large-scale scientific computing because it produces a well-conditioned basis for the Krylov subspace more quickly than the standard Arnoldi process. However, the resulting Hessenberg matrix is generally not similar to the one produced by the standard Arnoldi process, which can lead to delays or spike-like irregularities in convergence. In this paper, we introduce a modification of the randomized Arnoldi process that restores similarity with the Hessenberg matrix generated by the standard Arnoldi process. This is accomplished by enforcing orthogonality between the last Arnoldi vector and the previously generated subspace, which requires solving only one additional least-squares problem. When applied to eigenvalue problems and matrix function evaluations, the modified randomized Arnoldi process produces approximations that are identical to those obtained with the standard Arnoldi process. Numerical experiments demonstrate that our approach is as fast as the randomized Arnoldi process and as robust as the standard Arnoldi process.

Restoring similarity in randomized Krylov methods with applications to eigenvalue problems and matrix functions

TL;DR

This work addresses the lack of similarity between Hessenberg matrices produced by standard and randomized Arnoldi methods, which can disrupt convergence in eigenvalue computations and matrix-function evaluations. It introduces a similarity-restoring correction (RAC) that enforces orthogonality of the last Krylov vector via a least-squares step, yielding a decomposition with , ensuring is similar to the standard . The method is applied to a similarity-restoring randomized Krylov–Schur algorithm (SRR-KS) and its matrix-function counterpart, proving exact-equivalence to standard KS in exact arithmetic and establishing backward stability. Numerical experiments show that SRR-KS is as fast as randomized Krylov–Schur and as robust as the standard approach, with significant mitigation of convergence spikes in both eigenvalue problems and computations. Overall, the paper delivers a practical, stable, and scalable framework for reliable large-scale Krylov-based computations.

Abstract

The randomized Arnoldi process has been used in large-scale scientific computing because it produces a well-conditioned basis for the Krylov subspace more quickly than the standard Arnoldi process. However, the resulting Hessenberg matrix is generally not similar to the one produced by the standard Arnoldi process, which can lead to delays or spike-like irregularities in convergence. In this paper, we introduce a modification of the randomized Arnoldi process that restores similarity with the Hessenberg matrix generated by the standard Arnoldi process. This is accomplished by enforcing orthogonality between the last Arnoldi vector and the previously generated subspace, which requires solving only one additional least-squares problem. When applied to eigenvalue problems and matrix function evaluations, the modified randomized Arnoldi process produces approximations that are identical to those obtained with the standard Arnoldi process. Numerical experiments demonstrate that our approach is as fast as the randomized Arnoldi process and as robust as the standard Arnoldi process.
Paper Structure (33 sections, 3 theorems, 54 equations, 8 figures)

This paper contains 33 sections, 3 theorems, 54 equations, 8 figures.

Table of Contents

  1. Introduction
  2. A new variant of the randomized Arnoldi process
  3. Arnoldi and randomized Arnoldi process
  4. Arnoldi process
  5. Randomized Arnoldi process
  6. Restoring similarity
  7. Implications for linear systems
  8. Similarity-restoring randomized Arnoldi
  9. Application to eigenvalue problems
  10. Arnoldi, Krylov--Schur and randomized Krylov--Schur method
  11. Basic Arnoldi method
  12. Restarting with the Krylov--Schur method
  13. Randomized Krylov--Schur method
  14. A new randomized Krylov--Schur method that restores similarity
  15. Initialization
  16. ...and 18 more sections

Key Result

Theorem 2.2

\newlabelthm:OKD0 Given the Krylov decomposition eq:OKD, let $\widehat{Q}_{m}$ be an orthonormal basis of $\mathsf{range}(U_{m})$. Then $\widehat{H}_{m}$ and $\widehat{G}_{m} := \widehat{Q}_{m}^{\mathsf{H}}A\widehat{Q}_{m}$ are similar. Moreover, the following Krylov decomposition holds:

Figures (8)

  • Figure 1: Equivalence of one cycle of \ref{['algo']} (using similarity-restoring randomized Arnoldi) and one cycle of Krylov--Schur (using standard Arnoldi) applied to a Krylov decomposition (KD) of the form \ref{['eq:OKD']} and its associated orthonormal Krylov decomposition (AOKD), respectively.
  • Figure 1: Execution time (s) of Krylov--Schur (KS), \ref{['algo']} (SRR-KS) with Cholesky and LSQR inner least-squares solvers, and randomized Krylov--Schur (RKS) versus matrix sizes. From left to right, the eigenvalues are of type $f_{1}$ to $f_{4}$ in \ref{['eq:deffi']} for the non-Hermitian (top) and Hermitian (bottom) example matrix \ref{['eq:defAn']}. \newlabelfig:time0
  • Figure 2: Execution time (s) for KS (left), SRR-KS (middle) and RKS (right) on Hermitian (top) and non-Hermitian (bottom) problems. The X-label and Y-label are $\ell$ and $m$, the dimension of Krylov subspaces before and after restarting, respectively. The stopping criterion is when the residual norms are below $10^{-7}$. The RKS method applied to the non-Hermitian problem with $\ell=20$ and $m=30$ does not converge after 10000 matvecs. \newlabelfig:dMax0
  • Figure 3: Convergence history of residual for KS, SRR-KS and RKS on Hermitian (left) and non-Hermitian (right) problems with $\ell=20$ and $m=30$. The history of KS and SRR-KS are overlapped. \newlabelfig:hist0
  • Figure 4: Convergence history for the approximation of $f(A)b$ for the test matrix in \ref{['subsubsec:experiments-matfun-convergence']}, comparing standard Arnoldi \ref{['eqn:fAq-arnoldi-orth']}, randomized Arnoldi \ref{['eqn:fAq-arnoldi-rand']} and SRR-Arnoldi (\ref{['algo:matfun']}). The curves for standard Arnoldi and SRR-Arnoldi using Cholesky or LSQR with tolerance $10^{-12}$ are completely overlapping. Top: relative errors. Bottom: ratios with respect to the error of the standard Arnoldi approximation. \newlabelfig:fAb-comparison0
  • ...and 3 more figures

Theorems & Definitions (8)

  • Definition 2.1
  • Theorem 2.2
  • Proof 1: Proof of \ref{['thm:OKD']}
  • Lemma 3.1
  • Proof 2
  • Theorem 3.2
  • Proof 3
  • Remark 4.1