Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A note on the VC dimension of 1-dimensional GNNs

Noah Daniëls, Floris Geerts

TL;DR

It is extended to demonstrate that 1-dimensional GNNs with a single parameter have an infinite VC dimension for unbounded graphs, and it is shown that this also holds for GNNs using analytic non-polynomial activation functions, including the 1-dimensional GNNs that were recently shown to be as expressive as the 1-WL test.

Abstract

Graph Neural Networks (GNNs) have become an essential tool for analyzing graph-structured data, leveraging their ability to capture complex relational information. While the expressivity of GNNs, particularly their equivalence to the Weisfeiler-Leman (1-WL) isomorphism test, has been well-documented, understanding their generalization capabilities remains critical. This paper focuses on the generalization of GNNs by investigating their Vapnik-Chervonenkis (VC) dimension. We extend previous results to demonstrate that 1-dimensional GNNs with a single parameter have an infinite VC dimension for unbounded graphs. Furthermore, we show that this also holds for GNNs using analytic non-polynomial activation functions, including the 1-dimensional GNNs that were recently shown to be as expressive as the 1-WL test. These results suggest inherent limitations in the generalization ability of even the most simple GNNs, when viewed from the VC dimension perspective.

A note on the VC dimension of 1-dimensional GNNs

TL;DR

It is extended to demonstrate that 1-dimensional GNNs with a single parameter have an infinite VC dimension for unbounded graphs, and it is shown that this also holds for GNNs using analytic non-polynomial activation functions, including the 1-dimensional GNNs that were recently shown to be as expressive as the 1-WL test.

Abstract

Graph Neural Networks (GNNs) have become an essential tool for analyzing graph-structured data, leveraging their ability to capture complex relational information. While the expressivity of GNNs, particularly their equivalence to the Weisfeiler-Leman (1-WL) isomorphism test, has been well-documented, understanding their generalization capabilities remains critical. This paper focuses on the generalization of GNNs by investigating their Vapnik-Chervonenkis (VC) dimension. We extend previous results to demonstrate that 1-dimensional GNNs with a single parameter have an infinite VC dimension for unbounded graphs. Furthermore, we show that this also holds for GNNs using analytic non-polynomial activation functions, including the 1-dimensional GNNs that were recently shown to be as expressive as the 1-WL test. These results suggest inherent limitations in the generalization ability of even the most simple GNNs, when viewed from the VC dimension perspective.

Paper Structure

This paper contains 10 sections, 8 theorems, 15 equations, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graphs
  4. Graph neural networks
  5. The Weisfeiler--Leman algorithm
  6. Generalisation ability and VC dimension
  7. Formal statement of previous and new results
  8. Piecewise polynomial activation functions
  9. Analytic non-polynomial activation functions
  10. Conclusion

Key Result

Theorem 3.1

expressive1expressive2 Let $n \in \mathbb{N}_{>0}$. Then there exists a GNN in $\mathop{\mathrm{GNN}}\nolimits_{\textsl{pl}}(\mathcal{O}(n),n)$ such that for all $t\in[n]$ the vertex embedding computed in the $t$-th layer is equivalent to the coloring produced by the $$1$\textrm{-}\textsf{WL}$ test

Theorems & Definitions (12)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 3.4: Stronger version of Theorem \ref{['th:gnn_pp_infinity1']}
  • Theorem 3.5
  • Proposition 4.1: Analog of Proposition 3.5 in generalization1
  • proof
  • Lemma 5.1
  • proof
  • Proposition 5.1
  • ...and 2 more