BOLT: Block-Orthonormal Lanczos for Trace estimation of matrix functions

TL;DR

This work tackles the challenge of estimating traces of matrix functions under memory and restricted-access constraints. It introduces Block-Orthonormal SLQ (BOLT), a block-based Lanczos quadrature method that matches Hutch++ accuracy without requiring randomized SVD, and shows superior performance in flat-spectrum regimes. Building on BOLT, Subblock SLQ enables memory-efficient trace estimation using only small principal submatrices, enabling proxy KL divergence and Wasserstein-2 distance computations in singular or under-sampled settings. The authors provide rigorous theoretical guarantees (unbiasedness, variance bounds, localization) and demonstrate practical gains on high-dimensional problems, including MNIST regularization and HODLR-divergence estimation, with detailed FLOP- and memory-analysis. Overall, the framework delivers scalable, low-memory tools for trace-based divergences and distributional distances in large-scale covariance-structured problems.

Abstract

Efficient matrix trace estimation is essential for scalable computation of log-determinants, matrix norms, and distributional divergences. In many large-scale applications, the matrices involved are too large to store or access in full, making even a single matrix-vector (mat-vec) product infeasible. Instead, one often has access only to small subblocks of the matrix or localized matrix-vector products on restricted index sets. Hutch++ achieves optimal convergence rate but relies on randomized SVD and assumes full mat-vec access, making it difficult to apply in these constrained settings. We propose the Block-Orthonormal Stochastic Lanczos Quadrature (BOLT), which matches Hutch++ accuracy with a simpler implementation based on orthonormal block probes and Lanczos iterations. BOLT builds on the Stochastic Lanczos Quadrature (SLQ) framework, which combines random probing with Krylov subspace methods to efficiently approximate traces of matrix functions, and performs better than Hutch++ in near flat-spectrum regimes. To address memory limitations and partial access constraints, we introduce Subblock SLQ, a variant of BOLT that operates only on small principal submatrices. As a result, this framework yields a proxy KL divergence estimator and an efficient method for computing the Wasserstein-2 distance between Gaussians - both compatible with low-memory and partial-access regimes. We provide theoretical guarantees and demonstrate strong empirical performance across a range of high-dimensional settings.

Figures (12)

• Figure 1: Timing comparisons
• Figure 1: Empirical trace recovery versus sampling ratio $st/N$.
• Figure 1: Histogram of scaled smallest eigenvalue of $W(m,m)$ for $m = 50$.
• Figure 2: Relative error (KL divergence) vs. number of matrix–vector products (Scalar-Hutch vs. Block SLQ) vs. Hutch++
• Figure 3: BOLT outperforms Hutch++ in approximating the trace of $\mathrm{diag}(\mathrm{Unif}[1,2])^2$ (150 trials)

Theorems & Definitions (13)

• Theorem 2.1: Block SLQ Estimator
• Remark 1
• Lemma 3.1: Unbiased Subblock Trace Estimation
• Theorem 3.2: Exact Localization of Polynomial Filters on Principal Subblocks
• Remark 2
• Lemma 3.3: Subblock full‐rankness from Wishart sampling
• Definition 3.4: Proxy KL Divergence
• Proof 1
• Proof 2
• Proof 3