Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective

Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song, Wei Wang, Jiahao Zhang

TL;DR

This paper examines the computational capabilities of Graph Neural Networks (GNNs) through circuit complexity, proposing that GNNs with constant depth, poly(n) precision, and embedding size d = O(n) can be simulated by a uniform TC^0 circuit. It establishes that, unless TC^0 = NC^1, such GNNs cannot solve fundamental graph problems like graph connectivity and graph isomorphism, thereby revealing intrinsic expressivity limitations beyond WL-based analyses. The authors decompose GNN components (activations, graph convolutions, pooling, and MLP heads) and prove TC^0 bounds for each, then combine them to derive a global TC^0 bound for multi-layer GNNs with m = O(1). The results provide a principled framework for assessing GNN expressiveness and suggest that surpassing TC^0 requires architectural innovations or different computational paradigms, with implications for designing graph-reasoning models in practice.

Abstract

Graph Neural Networks (GNNs) have become the standard approach for learning and reasoning over relational data, leveraging the message-passing mechanism that iteratively propagates node embeddings through graph structures. While GNNs have achieved significant empirical success, their theoretical limitations remain an active area of research. Existing studies primarily focus on characterizing GNN expressiveness through Weisfeiler-Lehman (WL) graph isomorphism tests. In this paper, we take a fundamentally different approach by exploring the computational limitations of GNNs through the lens of circuit complexity. Specifically, we analyze the circuit complexity of common GNN architectures and prove that under constraints of constant-depth layers, linear or sublinear embedding sizes, and polynomial precision, GNNs cannot solve key problems such as graph connectivity and graph isomorphism unless $\mathsf{TC}^0 = \mathsf{NC}^1$. These results reveal the intrinsic expressivity limitations of GNNs behind their empirical success and introduce a novel framework for analyzing GNN expressiveness that can be extended to a broader range of GNN models and graph decision problems.

On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective

TL;DR

This paper examines the computational capabilities of Graph Neural Networks (GNNs) through circuit complexity, proposing that GNNs with constant depth, poly(n) precision, and embedding size d = O(n) can be simulated by a uniform TC^0 circuit. It establishes that, unless TC^0 = NC^1, such GNNs cannot solve fundamental graph problems like graph connectivity and graph isomorphism, thereby revealing intrinsic expressivity limitations beyond WL-based analyses. The authors decompose GNN components (activations, graph convolutions, pooling, and MLP heads) and prove TC^0 bounds for each, then combine them to derive a global TC^0 bound for multi-layer GNNs with m = O(1). The results provide a principled framework for assessing GNN expressiveness and suggest that surpassing TC^0 requires architectural innovations or different computational paradigms, with implications for designing graph-reasoning models in practice.

Abstract

Graph Neural Networks (GNNs) have become the standard approach for learning and reasoning over relational data, leveraging the message-passing mechanism that iteratively propagates node embeddings through graph structures. While GNNs have achieved significant empirical success, their theoretical limitations remain an active area of research. Existing studies primarily focus on characterizing GNN expressiveness through Weisfeiler-Lehman (WL) graph isomorphism tests. In this paper, we take a fundamentally different approach by exploring the computational limitations of GNNs through the lens of circuit complexity. Specifically, we analyze the circuit complexity of common GNN architectures and prove that under constraints of constant-depth layers, linear or sublinear embedding sizes, and polynomial precision, GNNs cannot solve key problems such as graph connectivity and graph isomorphism unless . These results reveal the intrinsic expressivity limitations of GNNs behind their empirical success and introduce a novel framework for analyzing GNN expressiveness that can be extended to a broader range of GNN models and graph decision problems.
Paper Structure (24 sections, 23 theorems, 28 equations)

This paper contains 24 sections, 23 theorems, 28 equations.

Table of Contents

  1. Introduction
  2. Related Work
  3. Limitations of Graph Neural Networks
  4. Circuit Complexity and Neural Networks
  5. Preliminary
  6. Notations.
  7. Floating Point Numbers
  8. Graph Neural Networks
  9. Circuit Complexity Classes
  10. Complexity of Graph Neural Networks
  11. Computing Activation Functions
  12. Computing Graph Convolution Matrices
  13. Computing Single Graph Neural Network Layer
  14. Computing Pooling Layer and Prediction Head
  15. Computing Multi-Layer Graph Neural Network
  16. ...and 9 more sections

Key Result

Lemma 3.4

We denote the number of digits as a positive integer $p$. If $p\leq \mathop{\mathrm{poly}}\nolimits(n)$, then:

Theorems & Definitions (74)

  • Definition 3.1: Floating Point Numbers ($\mathsf{FPN}$s), Definition 9 in chi24
  • Definition 3.2: Rounding, Definition 9 in chi24
  • Definition 3.3: $\mathsf{FPN}$ operations, page 5 on chi24
  • Lemma 3.4: Computing $\mathsf{FPN}$ operations with $\mathsf{TC}^0$ circuits, Lemma 10 and Lemma 11 in chi24
  • Lemma 3.5: Computing $\exp$ with $\mathsf{TC}^0$ circuits, Lemma 12 in chi24
  • Lemma 3.6: Computing square root with $\mathsf{TC}^0$ circuits, Lemma 12 in chi24
  • Lemma 3.7: Computing matrix multiplication with $\mathsf{TC}^0$ circuits, Lemma 4.2 in cll+24
  • Definition 3.8: ReLU
  • Definition 3.9: Leaky ReLU
  • Definition 3.10: Softmax
  • ...and 64 more