A sketch-and-select Arnoldi process

TL;DR

The paper addresses the cost- and conditioning-related bottlenecks of building Krylov bases for large-scale problems by introducing the sketch-and-select Arnoldi process, which uses randomized subspace embeddings $S$ to form a non-orthogonal basis while selecting at most $k$ projection directions via sparse least-squares on the sketched space. This yields a cost that scales linearly with the Krylov dimension while aiming to keep the basis well conditioned, improving the robustness of sketched Krylov methods such as sGMRES. The authors provide a comprehensive experimental study across 80 SuiteSparse matrices, compare multiple selection strategies (including simple top-$k$ coefficient selection and several CS-inspired methods like OMP and SP), and analyze the influence of the starting vector and ill-conditioning in sGMRES, supported by theoretical bounds on basis-condition growth. They show that the sketch-and-select variants often outperform truncated Arnoldi in conditioning and, when used as a basis constructor in sGMRES, can achieve higher accuracy on average, highlighting practical impact for large-scale, non-symmetric linear algebra and related applications. The work also identifies open questions about the role of singular-value distributions in convergence, suggesting future work on adaptive strategies, block variants, and deeper theoretical understanding of non-orthogonal Krylov constructions.

Abstract

A sketch-and-select Arnoldi process to generate a well-conditioned basis of a Krylov space at low cost is proposed. At each iteration the procedure utilizes randomized sketching to select a limited number of previously computed basis vectors to project out of the current basis vector. The computational cost grows linearly with the dimension of the Krylov space. The subset selection problem for the projection step is approximately solved with a number of heuristic algorithms and greedy methods used in statistical learning and compressive sensing.

Figures (12)

• Figure 1: Basic (non-optimized) MATLAB implementation of the sketch-and-select Arnoldi process. The function uses a sketch function that takes as input a vector with $N$ components and returns a sketch with $s\ll N$ components. In this variant, which we refer to as sketch + select pinv, the indices of Krylov basis vectors to project out are obtained by keeping the $k$ coefficients of the orthogonal projection which are largest in modulus. In an efficient implementation, the least squares problems with the sketched basis $S V_j$ should be solved by retaining and updating at each iteration a QR factorization of $S V_j$. \newlabelalg:ssa0
• Figure 1: Basis condition number growth of nine different methods for the matrix Norris/torso3, using a truncation parameter of $k=2$ (top) and $k=5$ (bottom).
• Figure 2: Basis sizes reached by the different methods, using a truncation parameter of $k=2$, with maximum basis size $m = 100$ and condition number bounded by $10^{12}$. Performance profiles (top) and basis sizes for the best performing methods (bottom).
• Figure 3: Basis sizes reached by the different methods, using a truncation parameter of $k=5$, with maximum basis size $m = 150$ and condition number bounded by $10^{12}$. Performance profiles (top) and basis sizes for the best performing methods (bottom).
• Figure 4: Basis sizes reached by the different methods, using a truncation parameter of $k=10$, with maximum basis size $m = 200$ and condition number bounded by $10^{12}$. Performance profiles (top) and basis sizes for the best performing methods (bottom).

Theorems & Definitions (4)

• Theorem 1
• Proof 1
• Corollary 2
• Proof 2