Near instance optimality of the Lanczos method for Stieltjes and related matrix functions
Marcel Schweitzer
TL;DR
This paper establishes a near instance optimality guarantee for the Lanczos method when computing $f(A)\mathbf{b}$ with $A$ Hermitian positive definite and $f$ in the Stieltjes class. By representing $f(A)\mathbf{b}$ as an integral of shifted inverses and applying a Woodbury-based low-rank update to the Lanczos blocks, it shows that the Lanczos error is within a constant factor of the best Krylov approximation, with an explicit bound depending on $\kappa(A)$ and the Lanczos coefficient $\beta_{m+1}$. The result extends to a related class of functions $f(z)=z g(z)$, where $g$ is Stieltjes, and is supported by numerical experiments that demonstrate sharpness and practical accuracy, often outperforming existing bounds. Consequently, one can analyze Lanczos performance for these functions via polynomial approximation on the eigenvalue spectrum, providing a more problem-dependent understanding of convergence and guiding expectations in applications such as fractional differential equations, network analysis, and Gaussian processes.
Abstract
Polynomial Krylov subspace methods are among the most widely used methods for approximating $f(A)b$, the action of a matrix function on a vector, in particular when $A$ is large and sparse. When $A$ is Hermitian positive definite, the Lanczos method is the standard choice of Krylov method, and despite being very simplistic in nature, it often outperforms other, more sophisticated methods. In fact, one often observes that the error of the Lanczos method behaves almost exactly as the error of the best possible approximation from the Krylov space (which is in general not efficiently computable). However, theoretical guarantees for the deviation of the Lanczos error from the optimal error are mostly lacking so far (except for linear systems and a few other special cases). We prove a rigorous bound for this deviation when $f$ belongs to the important class of Stieltjes functions (which, e.g., includes inverse fractional powers as special cases) and a related class (which contains, e.g., the square root and the shifted logarithm), thus providing a \emph{near instance optimality} guarantee. While the constants in our bounds are likely not optimal, they greatly improve over the few results that are available in the literature and resemble the actual behavior much better.