Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are Targeted Messages More Effective?

Martin Grohe, Eran Rosenbluth

TL;DR

The paper investigates whether 1-sided and 2-sided GNNs differ in expressive power, linking the problem to modal counting logic fragments. It proves that the modal fragment MC and the guarded fragment GC have the same expressive power on labelled undirected graphs, implying non-uniform 1-GNN and 2-GNN expressivity coincide, while in the uniform setting a strict separation emerges for SUM-aggregation. A hashing-based proof shows MFO+C and GFO+C are equally expressive, and the results extend to non-uniform settings with built-in numerical relations. The MEAN and MAX aggregations do not sustain a uniform separation, as 2-GNNs can be simulated by 1-GNNs under these aggregations. The work introduces novel techniques such as fast convergence and prime-hashing to transfer insights between logics and GNNs, with implications for when targeted messages yield greater predictive power.

Abstract

Graph neural networks (GNN) are deep learning architectures for graphs. Essentially, a GNN is a distributed message passing algorithm, which is controlled by parameters learned from data. It operates on the vertices of a graph: in each iteration, vertices receive a message on each incoming edge, aggregate these messages, and then update their state based on their current state and the aggregated messages. The expressivity of GNNs can be characterised in terms of certain fragments of first-order logic with counting and the Weisfeiler-Lehman algorithm. The core GNN architecture comes in two different versions. In the first version, a message only depends on the state of the source vertex, whereas in the second version it depends on the states of the source and target vertices. In practice, both of these versions are used, but the theory of GNNs so far mostly focused on the first one. On the logical side, the two versions correspond to two fragments of first-order logic with counting that we call modal and guarded. The question whether the two versions differ in their expressivity has been mostly overlooked in the GNN literature and has only been asked recently (Grohe, LICS'23). We answer this question here. It turns out that the answer is not as straightforward as one might expect. By proving that the modal and guarded fragment of first-order logic with counting have the same expressivity over labelled undirected graphs, we show that in a non-uniform setting the two GNN versions have the same expressivity. However, we also prove that in a uniform setting the second version is strictly more expressive.

Are Targeted Messages More Effective?

TL;DR

The paper investigates whether 1-sided and 2-sided GNNs differ in expressive power, linking the problem to modal counting logic fragments. It proves that the modal fragment MC and the guarded fragment GC have the same expressive power on labelled undirected graphs, implying non-uniform 1-GNN and 2-GNN expressivity coincide, while in the uniform setting a strict separation emerges for SUM-aggregation. A hashing-based proof shows MFO+C and GFO+C are equally expressive, and the results extend to non-uniform settings with built-in numerical relations. The MEAN and MAX aggregations do not sustain a uniform separation, as 2-GNNs can be simulated by 1-GNNs under these aggregations. The work introduces novel techniques such as fast convergence and prime-hashing to transfer insights between logics and GNNs, with implications for when targeted messages yield greater predictive power.

Abstract

Graph neural networks (GNN) are deep learning architectures for graphs. Essentially, a GNN is a distributed message passing algorithm, which is controlled by parameters learned from data. It operates on the vertices of a graph: in each iteration, vertices receive a message on each incoming edge, aggregate these messages, and then update their state based on their current state and the aggregated messages. The expressivity of GNNs can be characterised in terms of certain fragments of first-order logic with counting and the Weisfeiler-Lehman algorithm. The core GNN architecture comes in two different versions. In the first version, a message only depends on the state of the source vertex, whereas in the second version it depends on the states of the source and target vertices. In practice, both of these versions are used, but the theory of GNNs so far mostly focused on the first one. On the logical side, the two versions correspond to two fragments of first-order logic with counting that we call modal and guarded. The question whether the two versions differ in their expressivity has been mostly overlooked in the GNN literature and has only been asked recently (Grohe, LICS'23). We answer this question here. It turns out that the answer is not as straightforward as one might expect. By proving that the modal and guarded fragment of first-order logic with counting have the same expressivity over labelled undirected graphs, we show that in a non-uniform setting the two GNN versions have the same expressivity. However, we also prove that in a uniform setting the second version is strictly more expressive.
Paper Structure (20 sections, 33 theorems, 137 equations, 3 figures)

This paper contains 20 sections, 33 theorems, 137 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Feedforward Neural Networks
  4. Graphs and Signals
  5. Logics and Queries
  6. First-order Logic with Counting
  7. The Modal and Guarded Fragments
  8. Nonuniform Expressivity
  9. Proof of Theorem \ref{['thm:main-logic-intro']}
  10. Graph Neural Networks
  11. Expressivity and Logical Characterisations
  12. Uniform Separation
  13. SUM Aggregation
  14. The MEAN-MAX theorem
  15. Conclusions
  16. ...and 5 more sections

Key Result

Theorem 1.1

Figures (3)

  • Figure 1.1: 1-sided and 2-sided message passing in the graph $G=(\{u,v_1,v_2,v_3\},\{uv_1,uv_2,uv_3\})$. Every vertex $x$ has a state $\mathcal{f}(x)$. Messages are associated with oriented edges. In the 1-sided version (a), messages are a function of the state of the source vertex, in the 2-sided version (b) they are function of the states of both source and target.
  • Figure 4.1: Uniform expressivity
  • Figure 4.2: Nonuniform expressivity

Theorems & Definitions (52)

  • Theorem 1.1
  • Example 1.2
  • Theorem 1.3
  • Proposition 3.0
  • Example 3.1
  • Corollary 3.2
  • Lemma 3.3
  • Lemma 3.4: Grohe23a
  • Lemma 3.4
  • Lemma 3.5: Folklore, BarringtonIS90Hesse01HesseAB02
  • ...and 42 more