Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lost in Aggregation: On a Fundamental Expressivity Limit of Message-Passing Graph Neural Networks

Eran Rosenbluth

Abstract

We define a generic class of functions that captures most conceivable aggregations for Message-Passing Graph Neural Networks (MP-GNNs), and prove that any MP-GNN model with such aggregations induces only a polynomial number of equivalence classes on all graphs - while the number of non-isomorphic graphs is doubly-exponential (in number of vertices). Adding a familiar perspective, we observe that merely 2-iterations of Color Refinement (CR) induce at least an exponential number of equivalence classes, making the aforementioned MP-GNNs relatively infinitely weaker. Previous results state that MP-GNNs match full CR, however they concern a weak, 'non-uniform', notion of distinguishing-power where each graph size may required a different MP-GNN to distinguish graphs up to that size. Our results concern both distinguishing between non-equivariant vertices and distinguishing between non-isomorphic graphs.

Lost in Aggregation: On a Fundamental Expressivity Limit of Message-Passing Graph Neural Networks

Abstract

We define a generic class of functions that captures most conceivable aggregations for Message-Passing Graph Neural Networks (MP-GNNs), and prove that any MP-GNN model with such aggregations induces only a polynomial number of equivalence classes on all graphs - while the number of non-isomorphic graphs is doubly-exponential (in number of vertices). Adding a familiar perspective, we observe that merely 2-iterations of Color Refinement (CR) induce at least an exponential number of equivalence classes, making the aforementioned MP-GNNs relatively infinitely weaker. Previous results state that MP-GNNs match full CR, however they concern a weak, 'non-uniform', notion of distinguishing-power where each graph size may required a different MP-GNN to distinguish graphs up to that size. Our results concern both distinguishing between non-equivariant vertices and distinguishing between non-isomorphic graphs.
Paper Structure (10 sections, 8 theorems, 72 equations)

This paper contains 10 sections, 8 theorems, 72 equations.

Table of Contents

  1. Introduction
  2. New Results
  3. Preliminaries
  4. Limited by Aggregation
  5. Distinguishing-Power.
  6. Main Result
  7. Information Complexity.
  8. Comparison to Color Refinement
  9. Concluding Remarks
  10. Limited By Aggregation

Key Result

Lemma 3.2

Let $N$ be an MP-GNN, then

Theorems & Definitions (18)

  • Definition 3.1: Sublinear; Logarithmic, Aggregation
  • Example 3.1
  • Lemma 3.2
  • proof
  • Lemma 3.2
  • Lemma 3.3
  • proof
  • Remark 3.4
  • Theorem 3.4
  • proof
  • ...and 8 more