Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized-Accelerated FEAST: A Hybrid Approach for Large-Scale Eigenvalue Problems

Ayush Nadiger

TL;DR

RA-FEAST tackles the computational bottleneck of extracting partial eigenspaces for large SPD matrices in statistics by fusing contour-based FEAST with randomized warmstarts. The method introduces a two-phase approach (randomized warmstart followed by accelerated FEAST) and provides probabilistic warmstart bounds, a perturbation-stability analysis for inexact updates, and a simple complexity model. Empirical results on sparse graph Laplacians show dramatic speedups (up to ~38x) while preserving accuracy down to spectral resolvability limits, validating the theoretical guarantees. This hybrid framework enables fast, accurate spectral decompositions in high-dimensional statistical applications such as covariance estimation and kernel methods, with practical impact for large-scale data analysis.

Abstract

We present Randomized-Accelerated FEAST (RA-FEAST), a hybrid algorithm that combines contour-integration-based eigensolvers with randomized numerical linear algebra techniques for efficiently computing partial eigendecompositions of large-scale matrices arising in statistical applications. By incorporating randomized subspace initialization to enable aggressive quadrature reduction and truncated refinement iterations, our method achieves significant computational speedups (up to 38x on sparse graph Laplacian benchmarks at n = 8000) while maintaining high-accuracy approximations to the target eigenspace. We provide a probabilistic error bound for the randomized warmstart, a stability result for inexact FEAST iterations under general perturbations, and a simple complexity model characterizing the trade-off between initialization cost and solver speedup. Empirically, we demonstrate that RA-FEAST can be more than an order of magnitude faster than standard FEAST while preserving accuracy on sparse Laplacian problems representative of modern spectral methods in statistics.

Randomized-Accelerated FEAST: A Hybrid Approach for Large-Scale Eigenvalue Problems

TL;DR

RA-FEAST tackles the computational bottleneck of extracting partial eigenspaces for large SPD matrices in statistics by fusing contour-based FEAST with randomized warmstarts. The method introduces a two-phase approach (randomized warmstart followed by accelerated FEAST) and provides probabilistic warmstart bounds, a perturbation-stability analysis for inexact updates, and a simple complexity model. Empirical results on sparse graph Laplacians show dramatic speedups (up to ~38x) while preserving accuracy down to spectral resolvability limits, validating the theoretical guarantees. This hybrid framework enables fast, accurate spectral decompositions in high-dimensional statistical applications such as covariance estimation and kernel methods, with practical impact for large-scale data analysis.

Abstract

We present Randomized-Accelerated FEAST (RA-FEAST), a hybrid algorithm that combines contour-integration-based eigensolvers with randomized numerical linear algebra techniques for efficiently computing partial eigendecompositions of large-scale matrices arising in statistical applications. By incorporating randomized subspace initialization to enable aggressive quadrature reduction and truncated refinement iterations, our method achieves significant computational speedups (up to 38x on sparse graph Laplacian benchmarks at n = 8000) while maintaining high-accuracy approximations to the target eigenspace. We provide a probabilistic error bound for the randomized warmstart, a stability result for inexact FEAST iterations under general perturbations, and a simple complexity model characterizing the trade-off between initialization cost and solver speedup. Empirically, we demonstrate that RA-FEAST can be more than an order of magnitude faster than standard FEAST while preserving accuracy on sparse Laplacian problems representative of modern spectral methods in statistics.

Paper Structure

This paper contains 25 sections, 4 theorems, 27 equations, 1 figure, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our approach.
  3. Contributions.
  4. Organization.
  5. Background
  6. The FEAST Algorithm
  7. Randomized Subspace Methods
  8. Randomized-Accelerated FEAST
  9. Algorithm Description
  10. Phase 1: Randomized warmstart.
  11. Phase 2: Accelerated FEAST Refinement.
  12. Theoretical Analysis
  13. Assumptions and Notation
  14. Preparatory Results
  15. Warmstart Error Bound
  16. ...and 10 more sections

Key Result

Lemma 1

Let $Q_0$ be obtained by applying randomized subspace iteration to $B$ with oversampling $p$ and $q$ power iterations as in eq:rand-range. Then, with probability at least $1-\delta$, where $C_1(\delta) := 1 + \frac{p}{2} \ln\!\bigl(\frac{2 m_0}{\delta}\bigr)$. For brevity, define

Figures (1)

  • Figure 1: Performance of RA-FEAST on Sparse Graph Laplacians. (A) Computational speedup relative to Standard FEAST ($N_c=8$), showing gains of up to $38\times$. (B) Wall-clock time scaling (log-log scale). (C) Overhead analysis at $n=16,000$ shows the randomized warmstart (Phase 1) is negligible compared to the solver time (Phase 2). (D) Accuracy validation showing RA-FEAST maintains errors comparable to the baseline up to the limit of spectral resolvability.

Theorems & Definitions (9)

  • Lemma 1: Randomized range finder Halko2011
  • Lemma 2: Projector distance and Davis--Kahan
  • proof
  • Theorem 1: Warmstart projector error
  • proof
  • Remark 1: Parameter selection
  • Theorem 2: Stability of Perturbed FEAST
  • proof
  • Remark 2: Sources of Perturbation