Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recurrent Graph Neural Networks and Arithmetic Circuits

Timon Barlag, Vivian Holzapfel, Laura Strieker, Jonni Virtema, Heribert Vollmer

TL;DR

The model of recurrent arithmetic circuits is introduced, which can be seen as arithmetic analogues of sequential or logical circuits, which utilise so-called memory gates which are used to store data between iterations of the recurrent circuit.

Abstract

We characterise the computational power of recurrent graph neural networks (GNNs) in terms of arithmetic circuits over the real numbers. Our networks are not restricted to aggregate-combine GNNs or other particular types. Generalizing similar notions from the literature, we introduce the model of recurrent arithmetic circuits, which can be seen as arithmetic analogues of sequential or logical circuits. These circuits utilise so-called memory gates which are used to store data between iterations of the recurrent circuit. While (recurrent) GNNs work on labelled graphs, we construct arithmetic circuits that obtain encoded labelled graphs as real valued tuples and then compute the same function. For the other direction we construct recurrent GNNs which are able to simulate the computations of recurrent circuits. These GNNs are given the circuit-input as initial feature vectors and then, after the GNN-computation, have the circuit-output among the feature vectors of its nodes. In this way we establish an exact correspondence between the expressivity of recurrent GNNs and recurrent arithmetic circuits operating over real numbers.

Recurrent Graph Neural Networks and Arithmetic Circuits

TL;DR

The model of recurrent arithmetic circuits is introduced, which can be seen as arithmetic analogues of sequential or logical circuits, which utilise so-called memory gates which are used to store data between iterations of the recurrent circuit.

Abstract

We characterise the computational power of recurrent graph neural networks (GNNs) in terms of arithmetic circuits over the real numbers. Our networks are not restricted to aggregate-combine GNNs or other particular types. Generalizing similar notions from the literature, we introduce the model of recurrent arithmetic circuits, which can be seen as arithmetic analogues of sequential or logical circuits. These circuits utilise so-called memory gates which are used to store data between iterations of the recurrent circuit. While (recurrent) GNNs work on labelled graphs, we construct arithmetic circuits that obtain encoded labelled graphs as real valued tuples and then compute the same function. For the other direction we construct recurrent GNNs which are able to simulate the computations of recurrent circuits. These GNNs are given the circuit-input as initial feature vectors and then, after the GNN-computation, have the circuit-output among the feature vectors of its nodes. In this way we establish an exact correspondence between the expressivity of recurrent GNNs and recurrent arithmetic circuits operating over real numbers.
Paper Structure (17 sections, 19 theorems, 10 equations, 14 figures, 3 algorithms)

This paper contains 17 sections, 19 theorems, 10 equations, 14 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Recurrent Arithmetic Circuits
  4. Network Model
  5. Recurrent Graph Neural Networks
  6. Recurrent Circuit Graph Neural Networks
  7. Relations Between Recurrent Circuits and Recurrent GNNs
  8. Simulating Recurrent GNNs With Recurrent Arithmetic Circuits
  9. Simulating Recurrent Arithmetic Circuits With Recurrent C-GNNs
  10. Conclusion
  11. Supplementary Material
  12. Proofs of Section \ref{['sec:rec_arithmetic_circs']}
  13. Proofs of Section \ref{['sec:gnn_to_circ']}
  14. Proofs of Section \ref{['sec:circ_to_gnn']}
  15. Encoding
  16. ...and 2 more sections

Key Result

Theorem 1

Let $(f_n)_{n\in \mathbb{N}\xspace}$, $(f'_n)_{n\in \mathbb{N}\xspace}$ be two function families in $\text{rec}[\mathfrak{F}_s]\text{-}\mathfrak{F}$. Then $(f'_n)_{n\in \mathbb{N}\xspace} \circ (f_n)_{n\in \mathbb{N}\xspace} = \left(f'_{n'}\circ f_n \right)_{n\in\mathbb{N}\xspace}$, where $n'\in\mat

Figures (14)

  • Figure 1: Structure of a recurrent arithmetic circuit: $\text{in}_i$ are the input gates, $\text{aux}_i$ are the auxiliary memory gates, $\text{out}_i$ are the output gates and $V_\text{halt}$ the halting gates. Dashed edges are the recurrent edges of the set $E_\text{rec}$. The halting function is not depicted.
  • Figure 2: Illustration of recurrent circuit $\textit{rec-}C$ with halting circuit $C_\text{halt}$ computing the $x_1$-th Fibonacci number for $1 < x_1 \in \mathbb{N}\xspace$. The dashed arrows mark the recurrent edges and the bold gate $\textit{in}_1$ is the single halting gate, i. e. the gate whose value forms the input for $C_\text{halt}$ along with the iteration number $i$.
  • Figure 3: The different models of (recurrent) C-GNNs. One block stands for one layer of the different (recurrent) C-GNN models $\mathcal{N}\xspace$, dashed edges mark recurrence
  • Figure 4: Circuit $C_=$ computing $x_1 = x_2$ for inputs $x_1, x_2$ with depth 7 and size 16.
  • Figure 5: Circuit $C_\text{flag}$ that computes the flag whether circuit rec-$C$ has halted, dependent on the current iteration number $i$ and the values of the halting gates val$(V_\text{halt})$.
  • ...and 9 more figures

Theorems & Definitions (61)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Remark 1
  • Definition 8
  • Definition 9
  • ...and 51 more