Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse random hypergraphs: Non-backtracking spectra and community detection

Ludovic Stephan, Yizhe Zhu

TL;DR

To the best of the knowledge, this is the first provable and efficient spectral algorithm that achieves the conjectured threshold for HSBMs with r blocks generated according to a general symmetric probability tensor.

Abstract

We consider the community detection problem in a sparse $q$-uniform hypergraph $G$, assuming that $G$ is generated according to the Hypergraph Stochastic Block Model (HSBM). We prove that a spectral method based on the non-backtracking operator for hypergraphs works with high probability down to the generalized Kesten-Stigum detection threshold conjectured by Angelini et al. (2015). We characterize the spectrum of the non-backtracking operator for the sparse HSBM and provide an efficient dimension reduction procedure using the Ihara-Bass formula for hypergraphs. As a result, community detection for the sparse HSBM on $n$ vertices can be reduced to an eigenvector problem of a $2n\times 2n$ non-normal matrix constructed from the adjacency matrix and the degree matrix of the hypergraph. To the best of our knowledge, this is the first provable and efficient spectral algorithm that achieves the conjectured threshold for HSBMs with $r$ blocks generated according to a general symmetric probability tensor.

Sparse random hypergraphs: Non-backtracking spectra and community detection

TL;DR

To the best of the knowledge, this is the first provable and efficient spectral algorithm that achieves the conjectured threshold for HSBMs with r blocks generated according to a general symmetric probability tensor.

Abstract

We consider the community detection problem in a sparse -uniform hypergraph , assuming that is generated according to the Hypergraph Stochastic Block Model (HSBM). We prove that a spectral method based on the non-backtracking operator for hypergraphs works with high probability down to the generalized Kesten-Stigum detection threshold conjectured by Angelini et al. (2015). We characterize the spectrum of the non-backtracking operator for the sparse HSBM and provide an efficient dimension reduction procedure using the Ihara-Bass formula for hypergraphs. As a result, community detection for the sparse HSBM on vertices can be reduced to an eigenvector problem of a non-normal matrix constructed from the adjacency matrix and the degree matrix of the hypergraph. To the best of our knowledge, this is the first provable and efficient spectral algorithm that achieves the conjectured threshold for HSBMs with blocks generated according to a general symmetric probability tensor.
Paper Structure (53 sections, 41 theorems, 359 equations, 8 figures, 2 algorithms)

This paper contains 53 sections, 41 theorems, 359 equations, 8 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Definitions and notations for hypergraphs
  3. Non-backtracking operator for hypergraphs
  4. Hypergraph stochastic block model
  5. Main results
  6. Spectrum of the non-backtracking matrix
  7. Dimension reduction via the Ihara-Bass formula
  8. Spectral reconstruction algorithm
  9. Discussion
  10. Reconstruction in the HSBM
  11. Beyond the sparse setting
  12. Non-uniform hypergraphs
  13. Possible extensions
  14. Other spectral properties
  15. Proof overview and technical novelties
  16. ...and 38 more sections

Key Result

Theorem 1

Let $G$ be a hypergraph generated according to the HSBM with $m$ hyperedges, and $B$ be its non-backtracking matrix. Denote $|\lambda_1(B)|\geq |\lambda_2(B)|\geq \cdots \geq |\lambda_{qm}(B)|$ the eigenvalues of $B$ in decreasing order of modules. Under Assumption assumption:degree, when $n$ is lar

Figures (8)

  • Figure 1: A $3$-uniform hypergraph with $5$ vertices $\{1,\dots, 5\}$ and three hyperedges $e_1=\{ 1,2,3\}, e_2=\{ 2,3, 5\}, e_3=\{ 1,4,5\}$
  • Figure 2: $(x,e,y,f)$ is a non-backtracking walk of length $1$
  • Figure 3: Spectrum of a symmetric HSBM (see Example \ref{['ex:symmetric_HSBM']}) with $n = 6000$, $q = r = 4$. The parameters $c_\mathrm{in}$ and $c_{\mathrm{out}}$ have been chosen so that $d = 4$ and $\mu_2 = 2$. The outlier eigenvalues and the bulk circle have been outlined in red. The single eigenvalue close to $(q-1)d = 12$ and the three eigenvalues near $(q-1)\mu_2 = 6$ are clearly visible.
  • Figure 4: Scatter plot of the last $n$ entries for the second and third eigenvector of $\tilde{B}$ under the symmetric HSBM (see Example \ref{['ex:symmetric_HSBM']}) with $q=4$, $r=3$ and $n=20000$. The parameters $c_\mathrm{in}$ and $c_{\mathrm{out}}$ have been chosen so that $d = 4$ and $\mu_2 = 2$. The colors correspond to the actual label of each vertex. Despite overlap near $(0, 0)$ (which is expected due to the presence of isolated vertices and small connected components), the three communities are easily recognizable.
  • Figure 5: A Galton-Watson hypertree with $q=3$
  • ...and 3 more figures

Theorems & Definitions (84)

  • Definition 1: Hypergraph
  • Definition 2: Adjacency tensor of a uniform hypergraph
  • Definition 3: Adjacency matrix of a hypergraph
  • Definition 4: Non-backtracking operator for hypergraphs
  • Definition 5: Hypergraph stochastic block model
  • Remark 1
  • Example 1: Symmetric HSBM
  • Theorem 1: Spectrum of $B$
  • Remark 2: Erdős-Rényi hypergraphs
  • Lemma 1: Ihara-Bass formula for uniform hypergraphs
  • ...and 74 more