Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Improved and Generalised Analysis for Spectral Clustering

George Tyler, Luca Zanetti

TL;DR

The paper develops a generalized, eigenvalue-gap–based framework for understanding spectral clustering beyond the standard Laplacian setting, including Hermitian graph representations and digraphs. It proves a generalized structure theorem: if the bottom informative eigenvalues are well separated from the rest, the bottom eigenvectors align closely with linear combinations of given cluster indicators, with computable bounds. The authors extend this to hierarchical, multi-scale clustering via a recursive structure theorem and introduce a cyclic k-way expansion for digraphs, yielding tighter, more predictive bounds than prior work. Empirical results on geometric graphs, SBMs, and real-world directed networks validate the theory and show substantial improvements over existing bounds, accompanied by open-source code.

Abstract

We revisit the theoretical performances of Spectral Clustering, a classical algorithm for graph partitioning that relies on the eigenvectors of a matrix representation of the graph. Informally, we show that Spectral Clustering works well as long as the smallest eigenvalues appear in groups well separated from the rest of the matrix representation's spectrum. This arises, for example, whenever there exists a hierarchy of clusters at different scales, a regime not captured by previous analyses. Our results are very general and can be applied beyond the traditional graph Laplacian. In particular, we study Hermitian representations of digraphs and show Spectral Clustering can recover partitions where edges between clusters are oriented mostly in the same direction. This has applications in, for example, the analysis of trophic levels in ecological networks. We demonstrate that our results accurately predict the performances of Spectral Clustering on synthetic and real-world data sets.

An Improved and Generalised Analysis for Spectral Clustering

TL;DR

The paper develops a generalized, eigenvalue-gap–based framework for understanding spectral clustering beyond the standard Laplacian setting, including Hermitian graph representations and digraphs. It proves a generalized structure theorem: if the bottom informative eigenvalues are well separated from the rest, the bottom eigenvectors align closely with linear combinations of given cluster indicators, with computable bounds. The authors extend this to hierarchical, multi-scale clustering via a recursive structure theorem and introduce a cyclic k-way expansion for digraphs, yielding tighter, more predictive bounds than prior work. Empirical results on geometric graphs, SBMs, and real-world directed networks validate the theory and show substantial improvements over existing bounds, accompanied by open-source code.

Abstract

We revisit the theoretical performances of Spectral Clustering, a classical algorithm for graph partitioning that relies on the eigenvectors of a matrix representation of the graph. Informally, we show that Spectral Clustering works well as long as the smallest eigenvalues appear in groups well separated from the rest of the matrix representation's spectrum. This arises, for example, whenever there exists a hierarchy of clusters at different scales, a regime not captured by previous analyses. Our results are very general and can be applied beyond the traditional graph Laplacian. In particular, we study Hermitian representations of digraphs and show Spectral Clustering can recover partitions where edges between clusters are oriented mostly in the same direction. This has applications in, for example, the analysis of trophic levels in ecological networks. We demonstrate that our results accurately predict the performances of Spectral Clustering on synthetic and real-world data sets.

Paper Structure

This paper contains 29 sections, 16 theorems, 70 equations, 15 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Organisation
  4. Background
  5. A generalised Structure Theorem
  6. Experimental results
  7. Geometric random graphs
  8. Stochastic block models
  9. Real-world networks
  10. Digraph clustering
  11. Comparison with previous work
  12. Experimental results
  13. Directed stochastic block models
  14. Real-world directed networks
  15. Conclusion
  16. ...and 14 more sections

Key Result

Theorem 1

Let $M \in \mathbb{C}^{N \times N}$ be Hermitian and positive semidefinite with eigenvalues $0 \le \lambda_1 \le \cdots \le \lambda_N$ and corresponding orthonormal basis of eigenvectors $f_1,\dots,f_N$. Let $g_1,\dots,g_k \in \mathbb{C}^N$ be orthonormal and let $\gamma_i \coloneq g_i^* M g_i$$(1 \

Figures (15)

  • Figure 1: Four clusters generated by sampling points from a mixture of 4 Gaussians and the corresponding geometric graph (\ref{['fig:4gaussianclustersgraph']}). Notice how the smallest four eigenvalues come in pairs (\ref{['fig:4gaussianclusterseigenvalues']}).
  • Figure 2: Illustration of how $\sum_{i=1}^{k}\|f_i - \hat{g}_i\|^2$ is formed from orthogonal matrices.
  • Figure 3: Illustration of how $Q_k$ might have blocks with higher values.
  • Figure 5: Examples of directed cluster structures.
  • Figure 6: Comparison of the results given by Theorem \ref{['thm:digraph']} (green for (\ref{['eq:ours_ray']}) and orange for (\ref{['eq:ours_psi']})) and by laenen2020higher (red) for a cyclic DSBM at varying level of noise. The actual values are reported in blue. Averaged over 10 realisations.
  • ...and 10 more figures

Theorems & Definitions (28)

  • Theorem 1
  • Corollary 2
  • Lemma 3
  • Theorem 4: Recursive Structure Theorem
  • Corollary 5
  • Definition 1
  • Lemma 6
  • Lemma 7
  • Theorem 8: Structure Theorem for Digraphs
  • Theorem 9: laenen2020higher
  • ...and 18 more