A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering

TL;DR

This work addresses the leading eigenproblem for spectral clustering on large graphs by applying a distributed Block Chebyshev-Davidson method that leverages analytic spectrum bounds of the symmetric normalized Laplacian $A = I - D^{-1/2} S D^{-1/2}$ with eigenvalues in $[0,2]$. It develops a scalable 2D/1D partitioned framework incorporating distributed SpMM, Chebyshev polynomial filtering, and TSQR-based orthonormalization, achieving approximately a $\sqrt{p}$ speedup as the number of processes $p$ grows. Numerical results show competitive clustering quality compared to ARPACK and LOBPCG while delivering superior parallel scalability, and demonstrate the method's effectiveness on graphs with up to millions of nodes. The approach enables scalable spectral clustering for very large graphs, with practical impact on data mining and network analysis tasks that rely on leading Laplacian eigenvectors.

Abstract

We develop a distributed Block Chebyshev-Davidson algorithm to solve large-scale leading eigenvalue problems for spectral analysis in spectral clustering. First, the efficiency of the Chebyshev-Davidson algorithm relies on the prior knowledge of the eigenvalue spectrum, which could be expensive to estimate. This issue can be lessened by the analytic spectrum estimation of the Laplacian or normalized Laplacian matrices in spectral clustering, making the proposed algorithm very efficient for spectral clustering. Second, to make the proposed algorithm capable of analyzing big data, a distributed and parallel version has been developed with attractive scalability. The speedup by parallel computing is approximately equivalent to $\sqrt{p}$, where $p$ denotes the number of processes. {Numerical results will be provided to demonstrate its efficiency in spectral clustering and scalability advantage over existing eigensolvers used for spectral clustering in parallel computing environments.}

Figures (9)

• Figure 1: Illustration of A-Stationary 1.5D SpMM $U = AV$ when the number of processes is $p = 9$. Process $P(2,1)$ owns the submatrices $U[7], V[5]$, and $A[2,1]$.
• Figure 2: Comparisons of ARPACK, LOBPCG without preconditioning, and the Block Chebyshev-Davidson method (Bchdav) in clustering performance on graphs with 50 thousand nodes. ARPACK runs with tolerance $.1$ and $.01$. LOBPCG and Bchdav run with tolerance $.1$. In Bchdav, $k_b = 4$ and $m = 11$.
• Figure 3: Comparisons of ARPACK, LOBPCG without preconditioning, and the Block Chebyshev-Davidson method (Bchdav) in clustering performance on graphs with 200 thousand nodes. ARPACK runs with tolerance $.1$ and $.01$. LOBPCG and Bchdav run with tolerance $.1$. In Bchdav, $k_b = 4$ and $m = 11$.
• Figure 4: Comparisons of LOBPCG with and without preconditioning (AMG) in clustering performance on graphs with 200 thousand nodes. LOBPCG runs with tolerance $.1$.
• Figure 5: Scaling of parallel ARPACK and LOBPCG to compute $k = 64$ eigenvectors of the matrix LBOLBSV(SG)-1M with tolerance $0.01$.