Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized block-Krylov subspace methods for low-rank approximation of matrix functions

David Persson, Tyler Chen, Christopher Musco

TL;DR

This work develops a Krylov-aware low-rank approximation framework for matrix functions, improving upon naive combinations of randomized SVD with inner Krylov-based matvecs. By leveraging the block-Lanczos structure and a key Krylov-krylov lemma, the authors show that one can compute accurate low-rank representations of $f(A)$ with the same matvec budget as standard approaches, but with significantly better error properties. They derive comprehensive error bounds, including inexact projections, structural, and probabilistic guarantees, and specialize these results to the matrix exponential, illustrating exponential convergence under suitable polynomial approximations. Numerical experiments across exponential integrators, graph measures, quantum spin systems, and a log example demonstrate that the Krylov-aware method often yields superior accuracy compared to randomized SVD variants, while maintaining comparable computational cost. Overall, the paper provides a rigorous foundation for Krylov-aware low-rank approximations of matrix functions and demonstrates their practical value in diverse applications.

Abstract

The randomized SVD is a method to compute an inexpensive, yet accurate, low-rank approximation of a matrix. The algorithm assumes access to the matrix through matrix-vector products (matvecs). Therefore, when we would like to apply the randomized SVD to a matrix function, $f(A)$, one needs to approximate matvecs with $f(A)$ using some other algorithm, which is typically treated as a black-box. Chen and Hallman (SIMAX 2023) argued that, in the common setting where matvecs with $f(A)$ are approximated using Krylov subspace methods (KSMs), a more efficient low-rank approximation is possible if we open this black-box. They present an alternative approach that significantly outperforms the naive combination of KSMs with the randomized SVD, although the method lacked theoretical justification. In this work, we take a closer look at the method, and provide strong and intuitive error bounds that justify its excellent performance for low-rank approximation of matrix functions.

Randomized block-Krylov subspace methods for low-rank approximation of matrix functions

TL;DR

This work develops a Krylov-aware low-rank approximation framework for matrix functions, improving upon naive combinations of randomized SVD with inner Krylov-based matvecs. By leveraging the block-Lanczos structure and a key Krylov-krylov lemma, the authors show that one can compute accurate low-rank representations of with the same matvec budget as standard approaches, but with significantly better error properties. They derive comprehensive error bounds, including inexact projections, structural, and probabilistic guarantees, and specialize these results to the matrix exponential, illustrating exponential convergence under suitable polynomial approximations. Numerical experiments across exponential integrators, graph measures, quantum spin systems, and a log example demonstrate that the Krylov-aware method often yields superior accuracy compared to randomized SVD variants, while maintaining comparable computational cost. Overall, the paper provides a rigorous foundation for Krylov-aware low-rank approximations of matrix functions and demonstrates their practical value in diverse applications.

Abstract

The randomized SVD is a method to compute an inexpensive, yet accurate, low-rank approximation of a matrix. The algorithm assumes access to the matrix through matrix-vector products (matvecs). Therefore, when we would like to apply the randomized SVD to a matrix function, , one needs to approximate matvecs with using some other algorithm, which is typically treated as a black-box. Chen and Hallman (SIMAX 2023) argued that, in the common setting where matvecs with are approximated using Krylov subspace methods (KSMs), a more efficient low-rank approximation is possible if we open this black-box. They present an alternative approach that significantly outperforms the naive combination of KSMs with the randomized SVD, although the method lacked theoretical justification. In this work, we take a closer look at the method, and provide strong and intuitive error bounds that justify its excellent performance for low-rank approximation of matrix functions.

Paper Structure

This paper contains 25 sections, 12 theorems, 78 equations, 3 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Krylov-aware low-rank approximation
  4. The block-Lanczos algorithm
  5. The randomized SVD for matrix functions
  6. Krylov-aware low-rank approximation
  7. Error bounds
  8. Error bounds for inexact projections
  9. Structural bounds
  10. Probabilistic bounds
  11. Error bounds for Krylov aware low-rank approximation
  12. Example bounds for the matrix exponential
  13. Single-vector Krylov-aware low-rank approximation
  14. Numerical experiments
  15. Test matrices
  16. ...and 10 more sections

Key Result

Lemma 2.1

The approximation eq:lanczos_approximation is exact if $f\in\mathbb{P}_{s-1}$, and the approximation eq:lanczos_approximation_qf is exact if $f\in \mathbb{P}_{2s-1}$.

Figures (3)

  • Figure 1: Comparing relative error (\ref{['eq:relative_error']}) for the the approximations returned by \ref{['alg:krylow']} without truncation (Krylov aware (untruncated)), \ref{['alg:krylow']} with truncation back to rank $k$ (Krylov aware (truncated)), \ref{['alg:rsvd_matfun']}(randSVD), and \ref{['alg:rsvd']}(exact randSVD). The black line shows the optimal rank $k$ approximation relative Frobenius norm error. The rank parameter $k$ is visible as titles in the figures. In all experiments we set $\ell = k$.
  • Figure 2: Comparing relative error (\ref{['eq:relative_error']}) for the the approximations returned by \ref{['alg:krylow']} without truncation (Krylov aware (untruncated)), \ref{['alg:krylow']} with truncation back to rank $k$ (Krylov aware (truncated)), \ref{['alg:rsvd_matfun']}(randSVD), and \ref{['alg:rsvd']}(exact randSVD). The black line shows the optimal rank $k$ approximation relative Frobenius norm error. The rank parameter $k$ is visible as titles in the figures. In all experiments we set $\ell = k+5$.
  • Figure 3: Comparing relative error (\ref{['eq:relative_error']}) for the the approximations returned by \ref{['alg:krylow']} without truncation (Krylov aware (untruncated)), \ref{['alg:krylow']} with truncation back to rank $k$ (Krylov aware (truncated)), \ref{['alg:krylow_single']} without truncation (SV Krylov aware (untruncated)), \ref{['alg:krylow_single']} with truncation back to rank $k$ (SV Krylov aware (truncated)). The black line shows the optimal rank $k$ approximation relative Frobenius norm error. The rank parameter $k$ is visible as titles in the figures.

Theorems & Definitions (25)

  • Remark
  • Remark
  • Lemma 2.1: see e.g. hochbruck_lubich, golub_meurant_09, chen_hallman_23
  • Lemma 2.2: see e.g. chen_hallman_23
  • Lemma 2.3
  • proof
  • Remark
  • Theorem 3.1
  • proof
  • Corollary 3.2
  • ...and 15 more