Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sharp Bounds for Poly-GNNs and the Effect of Graph Noise

Luciano Vinas, Arash A. Amini

TL;DR

The paper studies semi-supervised node classification with poly-GNNs under CSBM, establishing a sharp, depth-invariant rate of class-separation in the final node representations. It develops a novel walk-based noise-decomposition framework and derives tight upper and lower bounds for the signal and noise components of the feature SNR, showing the leading rate scales as $ u_n^{1/2}$ regardless of depth $k$ (for large graphs). The analysis reveals dominant walk types that govern noise, explains how graph noise can overwhelm aggregation benefits, and clarifies even/odd-depth effects on feature propagation. The results imply that graph aggregation improves performance over feature-only baselines at rate $ u_n^{1/2}$, while warning against unbounded depth due to oversmoothing being a scale effect, not a SNR gain.

Abstract

We investigate the classification performance of graph neural networks with graph-polynomial features, poly-GNNs, on the problem of semi-supervised node classification. We analyze poly-GNNs under a general contextual stochastic block model (CSBM) by providing a sharp characterization of the rate of separation between classes in their output node representations. A question of interest is whether this rate depends on the depth of the network $k$, i.e., whether deeper networks can achieve a faster separation? We provide a negative answer to this question: for a sufficiently large graph, a depth $k > 1$ poly-GNN exhibits the same rate of separation as a depth $k=1$ counterpart. Our analysis highlights and quantifies the impact of ``graph noise'' in deep GNNs and shows how noise in the graph structure can dominate other sources of signal in the graph, negating any benefit further aggregation provides. Our analysis also reveals subtle differences between even and odd-layered GNNs in how the feature noise propagates.

Sharp Bounds for Poly-GNNs and the Effect of Graph Noise

TL;DR

The paper studies semi-supervised node classification with poly-GNNs under CSBM, establishing a sharp, depth-invariant rate of class-separation in the final node representations. It develops a novel walk-based noise-decomposition framework and derives tight upper and lower bounds for the signal and noise components of the feature SNR, showing the leading rate scales as regardless of depth (for large graphs). The analysis reveals dominant walk types that govern noise, explains how graph noise can overwhelm aggregation benefits, and clarifies even/odd-depth effects on feature propagation. The results imply that graph aggregation improves performance over feature-only baselines at rate , while warning against unbounded depth due to oversmoothing being a scale effect, not a SNR gain.

Abstract

We investigate the classification performance of graph neural networks with graph-polynomial features, poly-GNNs, on the problem of semi-supervised node classification. We analyze poly-GNNs under a general contextual stochastic block model (CSBM) by providing a sharp characterization of the rate of separation between classes in their output node representations. A question of interest is whether this rate depends on the depth of the network , i.e., whether deeper networks can achieve a faster separation? We provide a negative answer to this question: for a sufficiently large graph, a depth poly-GNN exhibits the same rate of separation as a depth counterpart. Our analysis highlights and quantifies the impact of ``graph noise'' in deep GNNs and shows how noise in the graph structure can dominate other sources of signal in the graph, negating any benefit further aggregation provides. Our analysis also reveals subtle differences between even and odd-layered GNNs in how the feature noise propagates.
Paper Structure (46 sections, 37 theorems, 200 equations, 4 figures)

This paper contains 46 sections, 37 theorems, 200 equations, 4 figures.

Table of Contents

  1. Introduction
  2. SSNC with GNNs
  3. CSBM and Noise Decompositions
  4. Noise Decomposition
  5. Walk Decomposition of Noise
  6. Paper Overview
  7. Technical Contributions
  8. Notation.
  9. Main result
  10. Informal statement
  11. Formal statement
  12. Signal bounds
  13. Noise bounds
  14. Previous Work
  15. Signal Analysis
  16. ...and 31 more sections

Key Result

Theorem 1

Let $(A,X)$ be generated from an $L$-class CSBM satisfying as:scale-as:sep with $\nu_n \gtrsim \log n$ and sufficiently large $n$. Then, for any $k\geq 1$, with high probability, for a constant $C$ independent of $n$ and $k$. Furthermore, with probability bounded away from zero, for a constant $c > 0$ independent of $n$ and $k$.

Figures (4)

  • Figure 1: Walks and their corresponding graphs for $n=5$.
  • Figure 2: Walks and their unique undirected edges and nodes
  • Figure 3: A walk sequence $\bm w$ with $r=6$ components of length $k=3$ each, and its corresponding undirected graph $G(\bm w)$. This walk sequence belongs to $\mathcal{N}_{r,t,v}$ defined in \ref{['eq:Nrtv']} with $r=6$, $t = |[\bm w]| = 15$ (number of unique edges) and $v = |\llbracket \bm w \rrbracket| = 16$ (number of unique vertices).
  • Figure 4: The unlabeled graph $G^*$ that all $G(\bm w), \bm w \in \mathcal{N}_*$ are isomorphic to, for $r=6$ and $k=4$. Its core star is colored red, while matched pairs of emanating branches are in black.

Theorems & Definitions (61)

  • Theorem 1: Informal
  • Theorem 2: Signal bounds
  • Theorem 3: Noise upper bound
  • Theorem 4: Noise lower bound
  • Theorem 5: Main result
  • proof : Proof of Theorem \ref{['thm:main_result']}
  • Lemma 1
  • Lemma 2
  • proof
  • Lemma 3
  • ...and 51 more