XTrace: Making the most of every sample in stochastic trace estimation

TL;DR

The paper tackles the implicit trace estimation problem for a square matrix $A$ accessed through matvecs by introducing XTrace and XNysTrace, two randomized estimators that leverage variance reduction and the exchangeability principle. It also introduces XDiag for diagonal estimation. A theoretical analysis describes estimator performance as a function of the spectrum of $A$, and empirical results show errors at a fixed matvec budget that are orders of magnitude smaller than Girard-Hutchinson and Hutch++ methods. Collectively, these methods enable more accurate trace and diagonal estimates for large-scale matrices with limited access, with practical impact for scientific computing and related applications.

Abstract

The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard-Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs.

Figures (2)

• Figure 1: Example figure using external image files.
• Figure 2: Example PGFPLOTS figure.

Theorems & Definitions (4)

• theorem 1: Mean Value Theorem
• corollary 1
• proof
• proof : Proof of main theorem