Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Networks and Arithmetic Circuits

Timon Barlag, Vivian Holzapfel, Laura Strieker, Jonni Virtema, Heribert Vollmer

TL;DR

The computational power of neural networks that follow the graph neural network (GNN) architecture is characterized, not restricted to aggregate-combine GNNs or other particular types, not restricted to aggregate-combine GNNs or other particular types.

Abstract

We characterize the computational power of neural networks that follow the graph neural network (GNN) architecture, not restricted to aggregate-combine GNNs or other particular types. We establish an exact correspondence between the expressivity of GNNs using diverse activation functions and arithmetic circuits over real numbers. In our results the activation function of the network becomes a gate type in the circuit. Our result holds for families of constant depth circuits and networks, both uniformly and non-uniformly, for all common activation functions.

Graph Neural Networks and Arithmetic Circuits

TL;DR

The computational power of neural networks that follow the graph neural network (GNN) architecture is characterized, not restricted to aggregate-combine GNNs or other particular types, not restricted to aggregate-combine GNNs or other particular types.

Abstract

We characterize the computational power of neural networks that follow the graph neural network (GNN) architecture, not restricted to aggregate-combine GNNs or other particular types. We establish an exact correspondence between the expressivity of GNNs using diverse activation functions and arithmetic circuits over real numbers. In our results the activation function of the network becomes a gate type in the circuit. Our result holds for families of constant depth circuits and networks, both uniformly and non-uniformly, for all common activation functions.
Paper Structure (12 sections, 11 theorems, 14 equations, 4 figures, 2 tables, 2 algorithms)

This paper contains 12 sections, 11 theorems, 14 equations, 4 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Neural Networks using Circuits
  4. Model of Computation
  5. Simulating C-GNNs with arithmetic circuits
  6. Simulating arithmetic circuits with C-GNNs
  7. Conclusion
  8. Appendix
  9. Proofs of Section \ref{['sec:prelim']}
  10. Proofs of Section \ref{['sec:model_of_comp']}
  11. Proofs of Section \ref{['sec:sim_cgnn_with_circ']}
  12. Proofs of Section \ref{['sec:sim_circ_with_cgnn']}

Key Result

Theorem 3.5$^\dagger$

Let $\mathcal{N}\xspace$ be a $(t\mathrm{FAC}^0_{\mathbb{R}\xspace^k}[\mathcal{A}]\xspace, \{\mathrm{id}\})$-GNN. Then there exists an $\mathrm{FAC}^0_{\mathbb{R}\xspace^k}[\mathcal{A}]\xspace$-circuit family $\mathcal{C}$, such that for all labeled graphs $\mathfrak{G}\xspace$ the following holds:

Figures (4)

  • Figure 1: Example illustrating the proof of Theorem \ref{['thm:circ-gnn-without-fnc']}.
  • Figure 2: Example illustrating the proof of Theorem \ref{['thm:circ-gnn-fnc']}.
  • Figure 3: Proof of Theorem \ref{['thm:cgnn_to_circ']}, where $\overline{v}_i$ are the feature vectors of the C-GNN, $\overline{u}_j$ the feature vectors of the neighbors of a node.
  • Figure 4: Example in the proof of Theorem \ref{['thm:circ-gnn-without-fnc']}

Theorems & Definitions (50)

  • Definition 2.1
  • Definition 2.2: AC-GNN, cf. Barcelo_2020
  • Definition 2.3
  • Remark 2.4
  • Definition 2.5
  • Remark 2.6
  • Definition 2.7
  • Definition 2.8
  • Definition 2.9
  • Remark 2.10
  • ...and 40 more