Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low degree conjecture implies sharp computational thresholds in stochastic block model

Jingqiu Ding, Yiding Hua, Lucas Slot, David Steurer

TL;DR

This work investigates the symmetric stochastic block model (SBM) near the Kesten-Stigum threshold through the lens of the extended low-degree conjecture. By constructing a correlation-preserving reduction and using cross-validated testing statistics, the authors provide rigorous evidence that no polynomial-time algorithm can weakly recover communities below KS (where $\varepsilon^2 d \le k^2$), while above KS, constant correlation recovery is achievable. They couple low-degree lower bounds for hypothesis testing to recovery and learnability results, yielding sharp computational-to-statistical gaps: below KS, even sub-exponential-time algorithms cannot reach $\Theta(n^{-0.5+\delta})$ recovery, and they extend these results to learning the edge-probability matrix and graphon function. The work also discusses concurrent lower bounds and the potential for diverging numbers of blocks under stronger conjectures, highlighting broader implications for average-case complexity in SBM-related tasks. Overall, the paper advances our understanding of computational barriers in community detection and parameter learning within SBM, offering a concrete phase-transition picture tied to the KS threshold.

Abstract

We investigate implications of the (extended) low-degree conjecture (recently formalized in [MW23]) in the context of the symmetric stochastic block model. Assuming the conjecture holds, we establish that no polynomial-time algorithm can weakly recover community labels below the Kesten-Stigum (KS) threshold. In particular, we rule out polynomial-time estimators that, with constant probability, achieve correlation with the true communities that is significantly better than random. Whereas, above the KS threshold, polynomial-time algorithms are known to achieve constant correlation with the true communities with high probability[Mas14,AS15]. To our knowledge, we provide the first rigorous evidence for the sharp transition in recovery rate for polynomial-time algorithms at the KS threshold. Notably, under a stronger version of the low-degree conjecture, our lower bound remains valid even when the number of blocks diverges. Furthermore, our results provide evidence of a computational-to-statistical gap in learning the parameters of stochastic block models. In contrast to prior work, which either (i) rules out polynomial-time algorithms for hypothesis testing with 1-o(1) success probability [Hopkins18, BBK+21a] under the low-degree conjecture, or (ii) rules out low-degree polynomials for learning the edge connection probability matrix [LG23], our approach provides stronger lower bounds on the recovery and learning problem. Our proof combines low-degree lower bounds from [Hopkins18, BBK+21a] with graph splitting and cross-validation techniques. In order to rule out general recovery algorithms, we employ the correlation preserving projection method developed in [HS17].

Low degree conjecture implies sharp computational thresholds in stochastic block model

TL;DR

This work investigates the symmetric stochastic block model (SBM) near the Kesten-Stigum threshold through the lens of the extended low-degree conjecture. By constructing a correlation-preserving reduction and using cross-validated testing statistics, the authors provide rigorous evidence that no polynomial-time algorithm can weakly recover communities below KS (where ), while above KS, constant correlation recovery is achievable. They couple low-degree lower bounds for hypothesis testing to recovery and learnability results, yielding sharp computational-to-statistical gaps: below KS, even sub-exponential-time algorithms cannot reach recovery, and they extend these results to learning the edge-probability matrix and graphon function. The work also discusses concurrent lower bounds and the potential for diverging numbers of blocks under stronger conjectures, highlighting broader implications for average-case complexity in SBM-related tasks. Overall, the paper advances our understanding of computational barriers in community detection and parameter learning within SBM, offering a concrete phase-transition picture tied to the KS threshold.

Abstract

We investigate implications of the (extended) low-degree conjecture (recently formalized in [MW23]) in the context of the symmetric stochastic block model. Assuming the conjecture holds, we establish that no polynomial-time algorithm can weakly recover community labels below the Kesten-Stigum (KS) threshold. In particular, we rule out polynomial-time estimators that, with constant probability, achieve correlation with the true communities that is significantly better than random. Whereas, above the KS threshold, polynomial-time algorithms are known to achieve constant correlation with the true communities with high probability[Mas14,AS15]. To our knowledge, we provide the first rigorous evidence for the sharp transition in recovery rate for polynomial-time algorithms at the KS threshold. Notably, under a stronger version of the low-degree conjecture, our lower bound remains valid even when the number of blocks diverges. Furthermore, our results provide evidence of a computational-to-statistical gap in learning the parameters of stochastic block models. In contrast to prior work, which either (i) rules out polynomial-time algorithms for hypothesis testing with 1-o(1) success probability [Hopkins18, BBK+21a] under the low-degree conjecture, or (ii) rules out low-degree polynomials for learning the edge connection probability matrix [LG23], our approach provides stronger lower bounds on the recovery and learning problem. Our proof combines low-degree lower bounds from [Hopkins18, BBK+21a] with graph splitting and cross-validation techniques. In order to rule out general recovery algorithms, we employ the correlation preserving projection method developed in [HS17].

Paper Structure

This paper contains 46 sections, 21 theorems, 75 equations, 3 algorithms.

Table of Contents

  1. Introduction
  2. Kesten-Stigum threshold in the symmetric stochastic block model
  3. Rigorous evidence for average case complexity
  4. The low-degree method for hypothesis testing
  5. Low-degree recovery lower bound
  6. Implications for learning stochastic block model
  7. Main result
  8. Computational lower bound for weak recovery in stochastic block model
  9. Concurrent work
  10. Computational lower bound for learning stochastic block model
  11. Preliminaries
  12. Low-degree framework
  13. Low-degree likelihood ratio lower bound in the SBM
  14. Extended low-degree hypothesis
  15. Organization
  16. ...and 31 more sections

Key Result

Theorem 2.1

Let $k, d\in \varmathbb N^+$ be such that $k\leqslant O(1), d\leqslant n^{o(1)}$. Assume that for any $d'\in \varmathbb N^+$ such that $0.999 d\leqslant d'\leqslant d$, Conjecture conj:low-degree holds with distribution $P$ given by $\text{SSBM}(n,\frac{d'}{n},\varepsilon,k)$ and distribution $Q$ gi

Theorems & Definitions (46)

  • Definition 1.1: Symmetric $k$-stochastic block model $\text{SSBM}(n,\frac{d}{n},\varepsilon,k)$
  • Definition 1.2: Recovery rate and weak recovery in the SBM
  • Conjecture 1.3: Low-degree conjecture
  • Theorem 2.1: Computational lower bound below the KS threshold, see \ref{['thm:full-main-theorem-weak-recovery']} for the full statement
  • Theorem 2.2: Computational lower bound for diverging number of blocks
  • Definition 2.3: Edge connection probability matrix for the SBM
  • Definition 2.4: Graphon in the symmetric SBM
  • Definition 2.5: Gromov-Wasserstein distance between graphons
  • Theorem 2.6: Computational lower bound for learning block graphon function
  • Theorem 3.1: Low-degree lower bound for SBM, Thm. 2.20 in bandeira2021spectral
  • ...and 36 more