Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Homomorphism Expressivity of Spectral Invariant Graph Neural Networks

Jingchu Gai, Yiheng Du, Bohang Zhang, Haggai Maron, Liwei Wang

TL;DR

This work provides a complete characterization of the expressive power of spectral invariant GNNs through the framework of graph homomorphism expressivity. The authors prove that, at depth d, spectral invariant GNNs can distinguish exactly the class of graphs whose parallel-tree depth is at most d, yielding a precise hierarchy between spectral methods and WL-based tests such as 1-WL and 2-FWL. They show that refinement via eigen-projections and multiple iterations strictly increases expressivity, and quantify subgraph counting capabilities (e.g., all cycles and short paths up to seven vertices). Extensions to higher-order and symmetric-power variants further generalize the framework, with experiments verifying the theoretical predictions on synthetic and molecular datasets. The results clarify when spectral invariants add power to GNNs and offer a roadmap for constructing architectures with desired expressivity for graph-structured tasks.

Abstract

Graph spectra are an important class of structural features on graphs that have shown promising results in enhancing Graph Neural Networks (GNNs). Despite their widespread practical use, the theoretical understanding of the power of spectral invariants -- particularly their contribution to GNNs -- remains incomplete. In this paper, we address this fundamental question through the lens of homomorphism expressivity, providing a comprehensive and quantitative analysis of the expressive power of spectral invariants. Specifically, we prove that spectral invariant GNNs can homomorphism-count exactly a class of specific tree-like graphs which we refer to as parallel trees. We highlight the significance of this result in various contexts, including establishing a quantitative expressiveness hierarchy across different architectural variants, offering insights into the impact of GNN depth, and understanding the subgraph counting capabilities of spectral invariant GNNs. In particular, our results significantly extend Arvind et al. (2024) and settle their open questions. Finally, we generalize our analysis to higher-order GNNs and answer an open question raised by Zhang et al. (2024).

Homomorphism Expressivity of Spectral Invariant Graph Neural Networks

TL;DR

This work provides a complete characterization of the expressive power of spectral invariant GNNs through the framework of graph homomorphism expressivity. The authors prove that, at depth d, spectral invariant GNNs can distinguish exactly the class of graphs whose parallel-tree depth is at most d, yielding a precise hierarchy between spectral methods and WL-based tests such as 1-WL and 2-FWL. They show that refinement via eigen-projections and multiple iterations strictly increases expressivity, and quantify subgraph counting capabilities (e.g., all cycles and short paths up to seven vertices). Extensions to higher-order and symmetric-power variants further generalize the framework, with experiments verifying the theoretical predictions on synthetic and molecular datasets. The results clarify when spectral invariants add power to GNNs and offer a roadmap for constructing architectures with desired expressivity for graph-structured tasks.

Abstract

Graph spectra are an important class of structural features on graphs that have shown promising results in enhancing Graph Neural Networks (GNNs). Despite their widespread practical use, the theoretical understanding of the power of spectral invariants -- particularly their contribution to GNNs -- remains incomplete. In this paper, we address this fundamental question through the lens of homomorphism expressivity, providing a comprehensive and quantitative analysis of the expressive power of spectral invariants. Specifically, we prove that spectral invariant GNNs can homomorphism-count exactly a class of specific tree-like graphs which we refer to as parallel trees. We highlight the significance of this result in various contexts, including establishing a quantitative expressiveness hierarchy across different architectural variants, offering insights into the impact of GNN depth, and understanding the subgraph counting capabilities of spectral invariant GNNs. In particular, our results significantly extend Arvind et al. (2024) and settle their open questions. Finally, we generalize our analysis to higher-order GNNs and answer an open question raised by Zhang et al. (2024).

Paper Structure

This paper contains 44 sections, 45 theorems, 40 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Spectral invariant GNNs
  4. Homomorphism expressivity
  5. Homomorphism Expressivity of Spectral Invariant GNNs
  6. Main results
  7. Implications
  8. Comparison with 2-FWL
  9. Hierarchy
  10. Subgraph Count
  11. Proof sketch
  12. Extensions
  13. Higher Order
  14. Symmetric Power
  15. Experiment
  16. ...and 29 more sections

Key Result

Theorem 3.3

For any $d\in \mathbb N$, the homomorphism expressivity of spectral invariant GNNs with $d$ iterations exists and can be characterized as follows: Specifically, the following properties hold:

Figures (5)

  • Figure 1: Illustration of a parallel edge with endpoints $(u,v)$ in (a) and a parallel tree with its skeleton on the right in (b).
  • Figure 2: A counterexample graph in ${\mathcal{F}}^{2-\mathsf{FWL}}\backslash{\mathcal{F}}^{\mathsf{Spec},(\infty)}$.
  • Figure 3: Counterexample for \ref{['cor:2_iteration']} and \ref{['cor:k+1_iteration']}
  • Figure 4: Hierarchy of spectral invariant GNN (abbreviated as Spectral IGN) and other mainstream GNNs. Each arrow points to the strictly stronger architecture.
  • Figure 5: Counterexample for \ref{['thm:eigenvalues_homomorphic_expressivity']}

Theorems & Definitions (113)

  • Definition 2.1
  • Definition 3.1: Parallel Edge
  • Definition 3.2: Parallel Tree
  • Theorem 3.3
  • Corollary 3.4
  • proof
  • Corollary 3.5
  • proof
  • Remark 3.6
  • Corollary 3.7
  • ...and 103 more