Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A novel Krylov subspace method for approximating Fréchet derivatives of large-scale matrix functions

Daniel Kressner, Peter Oehme

TL;DR

This work tackles efficiently computing the action of the Fréchet derivative $L_f(A,E)$ on a vector $\mathbf{b}$ for large matrix functions $f(A)$. It introduces a modified Krylov subspace method based on a block-triangular-preserving Arnoldi scheme, with a convergence analysis that ties the error to the best polynomial approximation of $f'$ on the numerical range $W(A)$. A separable-orthogonalization variant is proposed to reduce computational cost while maintaining accuracy, and the method is validated through numerical experiments on network centrality sensitivities and a PDE parameter-fitting problem, demonstrating fast convergence and practical runtimes. The approach enables reliable sensitivity analysis and gradient-based optimization involving matrix functions in large-scale settings.

Abstract

We present a novel Krylov subspace method for approximating $L_f(A, E) \vc{b}$, the matrix-vector product of the Fréchet derivative $L_f(A, E)$ of a large-scale matrix function $f(A)$ in direction $E$, a task that arises naturally in the sensitivity analysis of quantities involving matrix functions, such as centrality measures for networks. It also arises in the context of gradient-based methods for optimization problems that feature matrix functions, e.g., when fitting an evolution equation to an observed solution trajectory. In principle, the well-known identity \[ f\left( \begin{bmatrix} A & E \\ 0 & A \end{bmatrix} \right) \begin{bmatrix} 0 \\ \vc{b} \end{bmatrix} = \begin{bmatrix} L_f(A, E) \vc{b} \\ f(A) \vc{b} \end{bmatrix}, \] allows one to directly apply any standard Krylov subspace method, such as the Arnoldi algorithm, to address this task. However, this comes with the major disadvantage that the involved block triangular matrix has unfavorable spectral properties, which impede the convergence analysis and, to a certain extent, also the observed convergence. To avoid these difficulties, we propose a novel modification of the Arnoldi algorithm that aims at better preserving the block triangular structure. In turn, this allows one to bound the convergence of the modified method by the best polynomial approximation of the derivative $f^\prime$ on the numerical range of $A$. Several numerical experiments illustrate our findings.

A novel Krylov subspace method for approximating Fréchet derivatives of large-scale matrix functions

TL;DR

This work tackles efficiently computing the action of the Fréchet derivative on a vector for large matrix functions . It introduces a modified Krylov subspace method based on a block-triangular-preserving Arnoldi scheme, with a convergence analysis that ties the error to the best polynomial approximation of on the numerical range . A separable-orthogonalization variant is proposed to reduce computational cost while maintaining accuracy, and the method is validated through numerical experiments on network centrality sensitivities and a PDE parameter-fitting problem, demonstrating fast convergence and practical runtimes. The approach enables reliable sensitivity analysis and gradient-based optimization involving matrix functions in large-scale settings.

Abstract

We present a novel Krylov subspace method for approximating , the matrix-vector product of the Fréchet derivative of a large-scale matrix function in direction , a task that arises naturally in the sensitivity analysis of quantities involving matrix functions, such as centrality measures for networks. It also arises in the context of gradient-based methods for optimization problems that feature matrix functions, e.g., when fitting an evolution equation to an observed solution trajectory. In principle, the well-known identity allows one to directly apply any standard Krylov subspace method, such as the Arnoldi algorithm, to address this task. However, this comes with the major disadvantage that the involved block triangular matrix has unfavorable spectral properties, which impede the convergence analysis and, to a certain extent, also the observed convergence. To avoid these difficulties, we propose a novel modification of the Arnoldi algorithm that aims at better preserving the block triangular structure. In turn, this allows one to bound the convergence of the modified method by the best polynomial approximation of the derivative on the numerical range of . Several numerical experiments illustrate our findings.
Paper Structure (10 sections, 3 theorems, 25 equations, 13 figures, 2 tables, 2 algorithms)

This paper contains 10 sections, 3 theorems, 25 equations, 13 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Low-rank directions
  3. Krylov subspace approximation to $L_f(A,E) \mathbf{b}$
  4. Preliminaries
  5. Modified Arnoldi algorithm: Basic version
  6. Modified Arnoldi algorithm: Separate orthogonalization
  7. Convergence analysis
  8. Numerical Experiments
  9. Centrality Measure Sensitivity in Network Analysis
  10. Parameter Fitting for the 2D Parametric Heat Equation

Key Result

Theorem 1.2

Given $A \in \mathbb C^{n\times n}$ such that $L_f(A, \cdot)$ is well defined, $\mathbf{b}, \mathbf{c} \in \mathbb C^n$ and $E \in \mathbb C^{n\times n}$, it holds that $\mathbf{c}^* L_f(A, E) \mathbf{b} = \langle L_f(A,\mathbf{c} \mathbf{b}^*), E \rangle$.

Figures (13)

  • Figure 1: Error vs. iterations of four different methods to approximate $L_f(A, E) \mathbf{b}$ for \ref{['ex:sqrtm']}. The dashed curve corresponds to the Arnoldi method for approximating $f(A) \mathbf{b}$ and is shown for reference only.
  • Figure 2: Convergence of $S^{(TN)}_{ij}$ for Air500
  • Figure 3: Convergence of $S^{(TN)}_{ij}$ for Autobahn
  • Figure 4: Convergence of $S^{(TN)}_{ij}$ for Pajek/USPowerGrid
  • Figure 5: Convergence of $S^{(TN)}_{ij}$ for SNAP/as-735
  • ...and 8 more figures

Theorems & Definitions (7)

  • Example 1.1
  • Theorem 1.2
  • proof
  • Lemma 2.1
  • proof
  • Theorem 2.2
  • proof