FlexTrace: Exchangeable Randomized Trace Estimation for Matrix Functions

TL;DR

This article introduces a novel trace estimator, FlexTrace, an exchangeable, single-pass method that estimates ${\rm tr}(f({\bf A}))$ solely using matvecs with ${\bf A}$.

Abstract

We consider the task of estimating the trace of a matrix function, ${\rm tr}(f({\bf A}))$, of a large symmetric positive semi-definite matrix ${\bf A}$. This problem arises in multiple applications, including kernel methods and inverse problems. A key challenge across existing trace estimation methods is the need for matrix-vector products (matvecs) with $f({\bf A})$, which can be very expensive. In this article, we introduce a novel trace estimator, FlexTrace, an exchangeable, single-pass method that estimates ${\rm tr}(f({\bf A}))$ solely using matvecs with ${\bf A}$. We consider the case where $f$ is an operator monotone matrix function with $f(0)=0$, which includes functions such as $\log(1+x)$ and $x^{1/2}$, and derive probabilistic bounds showcasing the theoretical advantages of FlexTrace. Numerical experiments across synthetic examples and application domains demonstrate that FlexTrace provides substantially more accurate estimates of the trace of $f({\bf A})$ compared to existing methods.

Figures (9)

• Figure 1: Demonstration of FlexTrace (alg:fast_version) on synthetic test matrices.
• Figure 2: Demonstration of FlexTrace (alg:fast_version) on the Poly matrix.
• Figure 3: Comparison of FlexTrace with existing methods for $f(x)=\log(1+x)$ with Exp (left) and $f(x) = \sqrt{x}$ with Poly (right).
• Figure 4: Comparison of FlexTrace, FunNys, and RandSVD for nuclear norm estimation of $\mathbf{{X}}$
• Figure 5: Ground-truth initial condition $\boldsymbol{m}_{\rm true}$ (left) and terminal state (right)

Theorems & Definitions (34)

• Theorem 2.1
• Definition 2.1
• Proposition 2.2
• proof
• Theorem 3.1
• proof
• Theorem 3.2
• proof
• Theorem 3.3
• proof