Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The logic of rational graph neural networks

Sammy Khalife

TL;DR

The paper analyzes how activation functions shape the uniform expressivity of graph neural networks within the GC2 logic framework. It proves that ReLU GNNs uniformly express all GC2 queries, while rational-activation GNNs cannot uniformly express all GC2 queries, even under favorable assumptions; it introduces the RGC2 fragment and shows rational (and polynomial) GNNs can express it uniformly. The results highlight a nuanced separation between rational and piecewise-linear activations, contradicting the notion that rational activations universally maximize expressivity, and open pathways to identify rationally-expressible logical fragments that GNNs can reliably implement. Overall, the work connects logical expressivity, network activations, and color-refinement dynamics to map the capabilities and limits of modern GNN architectures.

Abstract

The expressivity of Graph Neural Networks (GNNs) can be described via appropriate fragments of the first order logic. Any query of the two variable fragment of graded modal logic (GC2) interpreted over labeled graphs can be expressed using a Rectified Linear Unit (ReLU) GNN whose size does not grow with graph input sizes [Barcelo & Al., 2020]. Conversely, a GNN expresses at most a query of GC2, for any choice of activation function. In this article, we prove that some GC2 queries of depth $3$ cannot be expressed by GNNs with any rational activation function. This shows that not all non-polynomial activation functions confer GNNs maximal expressivity, answering a open question formulated by [Grohe, 2021]. This result is also in contrast with the efficient universal approximation properties of rational feedforward neural networks investigated by [Boullé & Al., 2020]. We also present a rational subfragment of the first order logic (RGC2), and prove that rational GNNs can express RGC2 queries uniformly over all graphs.

The logic of rational graph neural networks

TL;DR

The paper analyzes how activation functions shape the uniform expressivity of graph neural networks within the GC2 logic framework. It proves that ReLU GNNs uniformly express all GC2 queries, while rational-activation GNNs cannot uniformly express all GC2 queries, even under favorable assumptions; it introduces the RGC2 fragment and shows rational (and polynomial) GNNs can express it uniformly. The results highlight a nuanced separation between rational and piecewise-linear activations, contradicting the notion that rational activations universally maximize expressivity, and open pathways to identify rationally-expressible logical fragments that GNNs can reliably implement. Overall, the work connects logical expressivity, network activations, and color-refinement dynamics to map the capabilities and limits of modern GNN architectures.

Abstract

The expressivity of Graph Neural Networks (GNNs) can be described via appropriate fragments of the first order logic. Any query of the two variable fragment of graded modal logic (GC2) interpreted over labeled graphs can be expressed using a Rectified Linear Unit (ReLU) GNN whose size does not grow with graph input sizes [Barcelo & Al., 2020]. Conversely, a GNN expresses at most a query of GC2, for any choice of activation function. In this article, we prove that some GC2 queries of depth cannot be expressed by GNNs with any rational activation function. This shows that not all non-polynomial activation functions confer GNNs maximal expressivity, answering a open question formulated by [Grohe, 2021]. This result is also in contrast with the efficient universal approximation properties of rational feedforward neural networks investigated by [Boullé & Al., 2020]. We also present a rational subfragment of the first order logic (RGC2), and prove that rational GNNs can express RGC2 queries uniformly over all graphs.
Paper Structure (11 sections, 10 theorems, 40 equations, 1 figure)

This paper contains 11 sections, 10 theorems, 40 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Neural Networks (GNNs)
  4. Logical background: GC2 and queries
  5. GNNs and color refinement
  6. Uniform expressivity of GNNs
  7. Rational GNNs have limited expressivity
  8. Towards a rational fragment of GC2
  9. Discussion and open problems
  10. Proof that piecewise linear GNNs (and AGG=SUM) are as expressive as GC2
  11. Logic background: general definitions

Key Result

Theorem 2.11

Let $d$ be a positive integer, and let $\xi$ be the output of a GNN after $d$ iterations. Then $\mathsf{cr}^{d}$ refines $\xi$, that is, for all graphs $G, G'$ and vertices $v \in V(G)$, $v' \in V(G')$, $\mathsf{cr}^{(d)}(G,v)=\mathsf{cr}^{d}(G',v') \implies \xi(G,v)=\xi(G',v')$.

Figures (1)

  • Figure 1: $T[k_1, \cdots, k_m]$

Theorems & Definitions (46)

  • Definition 2.1: (Symmetric) rational fractions/functions
  • Definition 2.2: Neural Network (NN)
  • Definition 2.3: Graph Neural Network (GNN)
  • Remark 2.4
  • Definition 2.5
  • Definition 2.6
  • Definition 2.7: Graded (or guarded) model logic with counting (GC) and GC2 barcelo2020logical
  • Example 2.8: barcelo2020logical
  • Definition 2.9: Embeddings and refinement
  • Definition 2.10: Color refinement
  • ...and 36 more