Randomized sketched TT-GMRES for linear systems with tensor structure

TL;DR

This paper tackles solving linear tensor equations in TT-format by marrying Krylov methods with randomized sketching. It introduces TT-sGMRES, a sketched TT-GMRES that uses structured sketchings, incomplete orthogonalization, and streaming TT rounding (STTA) to curb TT-rank growth and memory usage while preserving convergence. The approach demonstrates strong numerical performance on convection-diffusion PDEs and high-dimensional Markov chains, particularly when coupled with exponential-sum preconditioning and bounded-rank strategies. Overall, TT-sGMRES offers a scalable, memory-efficient alternative to ALS methods, with potential extensions to other tensor formats and architectures where low-rank tensor representations are viable.

Abstract

In the last decade, tensors have shown their potential as valuable tools for various tasks in numerical linear algebra. While most of the research has been focusing on how to compress a given tensor in order to maintain information as well as reducing the storage demand for its allocation, the solution of linear tensor equations is a less explored venue. Even if many of the routines available in the literature are based on alternating minimization schemes (ALS), we pursue a different path and utilize Krylov methods instead. The use of Krylov methods in the tensor realm is not new. However, these routines often turn out to be rather expensive in terms of computational cost and ALS procedures are preferred in practice. We enhance Krylov methods for linear tensor equations with a panel of diverse randomization-based strategies which remarkably increase the efficiency of these solvers making them competitive with state-of-the-art ALS schemes. The up-to-date randomized approaches we employ range from sketched Krylov methods with incomplete orthogonalization and structured sketching transformations to streaming algorithms for tensor rounding. The promising performance of our new solver for linear tensor equations is demonstrated by many numerical results.

Figures (6)

• Figure 1: Actual residuals of the vanilla and enhanced TT-sGMRES algorithms computed after each iteration for the PDE problem in section \ref{['sec:case-study-convection-diffusion']} with $d = 4$ and $n=34$.
• Figure 2: On the left, we report the runtime of the TT-GMRES and TT-sGMRES algorithms on convection-diffusion PDE problems of size $n=64$ across various dimensions $d$ and accuracy $10^{-4}$. On the right, we plot the maximum TT-ranks of the base vectors generated by TT-GMRES and TT-sGMRES with $d=6$, $n=64$ and ${\texttt{tol}=10^{-4}}$. In the right experiment, TT-GMRES converged in 1528.22 seconds with respect to the 80.03 seconds of TT-sGMRES.
• Figure 3: The above plots report the difference between the sketched residual and the true residual, for different values of $d$.
• Figure 4: On the left, the comparison between running TT-GMRES and TT-sGMRES for the Markov test case, with different values of $d$ and $n = 64$. On the right, the behavior of ranks of the basis vectors during the iterations, in the case $d = 5$.
• Figure 5: Runtime of TT-sPGMRES iteration for the convection-diffusion problem in section \ref{['sec:case-study-convection-diffusion']} with variable $n_i$ and $d = 5$; the target tolerance in this example is $10^{-8}$, and different values of maxrank are used. AMEn is run with standard parameters, and is taken from TT-Toolbox TT-Toolbox.

Theorems & Definitions (3)

• Definition 3.1: Random Gaussian TT-Tensor
• Remark 3.2
• Remark 5.1