Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved Spectral Density Estimation via Explicit and Implicit Deflation

Rajarshi Bhattacharjee, Rajesh Jayaram, Cameron Musco, Christopher Musco, Archan Ray

TL;DR

This work studies spectral density estimation for large symmetric matrices using matrix-vector queries. It introduces explicit deflation, where the top eigen-directions are approximately projected out via a block Krylov method before moment matching, achieving a bound on the Wasserstein-1 distance of the form $W_1(s_{oldsymbol{A}}, ilde{s}_{oldsymbol{A}}) \le \varepsilon\sigma_{l+1}(oldsymbol{A})$ with near-optimal query complexity; a matching lower bound shows this is tight up to polylog factors. It further analyzes Stochastic Lanczos Quadrature (SLQ) through an implicit deflation lens, proving bounds that parallel the explicit deflation results and introducing a variance-reduced SLQ variant to eliminate the mass misallocation from large eigenvalues. The paper also presents a lower bound and extensive empirical evaluation demonstrating that SLQ and VR-SLQ outperform explicit moment-matching methods, especially when the spectrum decays rapidly or contains a few dominant eigenvalues. Overall, the work provides refined, spectrum-adaptive guarantees for SDE via deflation and highlights practical algorithms that leverage Krylov techniques for accurate spectral statistics with few queries. The results advance understanding of how deflation—explicit or implicit—can significantly improve spectral-density estimates in the matrix-vector query model, with direct implications for Schatten-1 norm estimation and related spectral tasks in large-scale linear-algebra problems.

Abstract

We study algorithms for approximating the spectral density of a symmetric matrix $A$ that is accessed through matrix-vector product queries. By combining a previously studied Chebyshev polynomial moment matching method with a deflation step that approximately projects off the largest magnitude eigendirections of $A$ before estimating the spectral density, we give an $ε\cdotσ_\ell(A)$ error approximation to the spectral density in the Wasserstein-$1$ metric using $O(\ell\log n+ 1/ε)$ matrix-vector products, where $σ_\ell(A)$ is the $\ell^{th}$ largest singular value of $A$. In the common case when $A$ exhibits fast singular value decay, our bound can be much stronger than prior work, which gives an error bound of $ε\cdot ||A||_2$ using $O(1/ε)$ matrix-vector products. We also show that it is nearly tight: any algorithm giving error $ε\cdot σ_\ell(A)$ must use $Ω(\ell+1/ε)$ matrix-vector products. We further show that the popular Stochastic Lanczos Quadrature (SLQ) method matches the above bound, even though SLQ itself is parameter-free and performs no explicit deflation. This bound explains the strong practical performance of SLQ, and motivates a simple variant of SLQ that achieves an even tighter error bound. Our error bound for SLQ leverages an analysis that views it as an implicit polynomial moment matching method, along with recent results on low-rank approximation with single-vector Krylov methods. We use these results to show that the method can perform implicit deflation as part of moment matching.

Improved Spectral Density Estimation via Explicit and Implicit Deflation

TL;DR

This work studies spectral density estimation for large symmetric matrices using matrix-vector queries. It introduces explicit deflation, where the top eigen-directions are approximately projected out via a block Krylov method before moment matching, achieving a bound on the Wasserstein-1 distance of the form with near-optimal query complexity; a matching lower bound shows this is tight up to polylog factors. It further analyzes Stochastic Lanczos Quadrature (SLQ) through an implicit deflation lens, proving bounds that parallel the explicit deflation results and introducing a variance-reduced SLQ variant to eliminate the mass misallocation from large eigenvalues. The paper also presents a lower bound and extensive empirical evaluation demonstrating that SLQ and VR-SLQ outperform explicit moment-matching methods, especially when the spectrum decays rapidly or contains a few dominant eigenvalues. Overall, the work provides refined, spectrum-adaptive guarantees for SDE via deflation and highlights practical algorithms that leverage Krylov techniques for accurate spectral statistics with few queries. The results advance understanding of how deflation—explicit or implicit—can significantly improve spectral-density estimates in the matrix-vector query model, with direct implications for Schatten-1 norm estimation and related spectral tasks in large-scale linear-algebra problems.

Abstract

We study algorithms for approximating the spectral density of a symmetric matrix that is accessed through matrix-vector product queries. By combining a previously studied Chebyshev polynomial moment matching method with a deflation step that approximately projects off the largest magnitude eigendirections of before estimating the spectral density, we give an error approximation to the spectral density in the Wasserstein- metric using matrix-vector products, where is the largest singular value of . In the common case when exhibits fast singular value decay, our bound can be much stronger than prior work, which gives an error bound of using matrix-vector products. We also show that it is nearly tight: any algorithm giving error must use matrix-vector products. We further show that the popular Stochastic Lanczos Quadrature (SLQ) method matches the above bound, even though SLQ itself is parameter-free and performs no explicit deflation. This bound explains the strong practical performance of SLQ, and motivates a simple variant of SLQ that achieves an even tighter error bound. Our error bound for SLQ leverages an analysis that views it as an implicit polynomial moment matching method, along with recent results on low-rank approximation with single-vector Krylov methods. We use these results to show that the method can perform implicit deflation as part of moment matching.

Paper Structure

This paper contains 36 sections, 37 theorems, 208 equations, 1 figure, 5 algorithms.

Table of Contents

  1. Introduction
  2. Matrix-Vector Query Algorithms for SDE
  3. Existing Bounds
  4. Our Results
  5. Improved SDE via Moment Matching with Explicit Deflation
  6. Implicit Deflation Bounds for Stochastic Lanczos Quadrature
  7. Technical Overview
  8. Eigenvalue Deflation for SDE
  9. Error Analysis of Deflation
  10. SLQ and its Existing Analysis
  11. Moment Matching-Based Analysis of SLQ
  12. Implicit Deflation with SLQ
  13. Variance Reduced SLQ
  14. Roadmap
  15. Notation and Preliminaries
  16. ...and 21 more sections

Key Result

Theorem 1

Let $\mathbf{A} \in \mathbb{R}^{n \times n}$ be symmetric. For any $\epsilon \in (0,1)$, $l \in [n]$, and any constants $c_1,c_2 > 0$, Algorithm alg:sde performs $O\left(l\log n+\frac{b}{\epsilon} \right)$ matrix-vector products with $\mathbf{A}$ where $b=\max\left(1,\frac{1}{n\epsilon^2}\log^2 \fra

Figures (1)

  • Figure 1: Wasserstein-$1$ error of spectral density estimation approximation algorithms. In the figures above, we plot the Wasserstein-$1$ error of approximating the spectral density of several matrices using the algorithms presented in the paper and some baseline algorithms. We observe that in almost all cases, VR-SLQ algorithm outperforms all other SDE algorithms. We also observe that the variants of our deflation algorithm (Algorithm \ref{['alg:sde']}) generally outperform the corresponding baseline of CMM and KPM.

Theorems & Definitions (70)

  • Theorem 1: SDE with Explicit Deflation
  • Theorem 2: SDE Lower Bound
  • Corollary 1: Schatten-$1$ Norm Estimation
  • Theorem 3: SDE with SLQ
  • Theorem 4: SDE with Variance Reduced SLQ
  • Definition 2.1: Block Krylov Subspace
  • Definition 2.2: Chebyshev Series Expansion
  • Lemma 1: Angle between subspaces, generalization of Theorem 2.1 of drineas2018structural
  • proof
  • Lemma 2: Convergence of block Krylov subspace to top singular vectors
  • ...and 60 more