Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Graphop Analysis of Graph Neural Networks on Sparse Graphs: Generalization and Universal Approximation

Ofek Amran, Tom Gilat, Ron Levie

TL;DR

The paper develops a unified theory for graph neural networks that covers both sparse and dense graphs by embedding graphs as bofops within graphop theory and extending MPNNs to bofop-signals. It introduces the action metric to obtain compactness and Lipschitz continuity of MPNNs, and the DIDM-mover’s distance to capture the separation power of MPNNs via 1-WL extensions. A key achievement is proving universal approximation on the compact space of bofop-DIDMs and establishing generalization bounds for MPNNs in this setting, along with a hierarchical view of DIDM spaces separating graphons from general sparse graph limits. The framework also provides a canonical bridge between profile-based representations and operator-based graph limits, enabling both theoretical insights and potential practical aggregations that work across graph sizes. Together, these results advance the principled understanding of GNN generalization and expressivity in realistic, heterogeneous graph regimes.

Abstract

Generalization and approximation capabilities of message passing graph neural networks (MPNNs) are often studied by defining a compact metric on a space of input graphs under which MPNNs are Hölder continuous. Such analyses are of two varieties: 1) when the metric space includes graphs of unbounded sizes, the theory is only appropriate for dense graphs, and, 2) when studying sparse graphs, the metric space only includes graphs of uniformly bounded size. In this work, we present a unified approach, defining a compact metric on the space of graphs of all sizes, both sparse and dense, under which MPNNs are Hölder continuous. This leads to more powerful universal approximation theorems and generalization bounds than previous works. The theory is based on, and extends, a recent approach to graph limit theory called graphop analysis.

A Graphop Analysis of Graph Neural Networks on Sparse Graphs: Generalization and Universal Approximation

TL;DR

The paper develops a unified theory for graph neural networks that covers both sparse and dense graphs by embedding graphs as bofops within graphop theory and extending MPNNs to bofop-signals. It introduces the action metric to obtain compactness and Lipschitz continuity of MPNNs, and the DIDM-mover’s distance to capture the separation power of MPNNs via 1-WL extensions. A key achievement is proving universal approximation on the compact space of bofop-DIDMs and establishing generalization bounds for MPNNs in this setting, along with a hierarchical view of DIDM spaces separating graphons from general sparse graph limits. The framework also provides a canonical bridge between profile-based representations and operator-based graph limits, enabling both theoretical insights and potential practical aggregations that work across graph sizes. Together, these results advance the principled understanding of GNN generalization and expressivity in realistic, heterogeneous graph regimes.

Abstract

Generalization and approximation capabilities of message passing graph neural networks (MPNNs) are often studied by defining a compact metric on a space of input graphs under which MPNNs are Hölder continuous. Such analyses are of two varieties: 1) when the metric space includes graphs of unbounded sizes, the theory is only appropriate for dense graphs, and, 2) when studying sparse graphs, the metric space only includes graphs of uniformly bounded size. In this work, we present a unified approach, defining a compact metric on the space of graphs of all sizes, both sparse and dense, under which MPNNs are Hölder continuous. This leads to more powerful universal approximation theorems and generalization bounds than previous works. The theory is based on, and extends, a recent approach to graph limit theory called graphop analysis.
Paper Structure (129 sections, 54 theorems, 241 equations, 1 figure)

This paper contains 129 sections, 54 theorems, 241 equations, 1 figure.

Table of Contents

  1. Introduction
  2. A basic apparoach to generalization and universality.
  3. Limitations in previous results.
  4. Our contribution.
  5. Related Work
  6. Background
  7. Basic Definitions
  8. Basic Notations.
  9. Graphs.
  10. Analysis.
  11. MPNNs on Graphs
  12. DIDM-Mover's Distance
  13. IDMs and DIDMs.
  14. DIDM-Mover's Distance.
  15. MPNNs on General DIDMs
  16. ...and 114 more sections

Key Result

Theorem 3.3

Let $A$ be a bofop over the Borel probability space $(\Omega,\mu)$. Then, there exists a unique measurable family of measures $(\nu_x)_{x\in\Omega}$, called fibers, where each $\nu_x$ is a Borel measure over $\Omega$, that satisfy the following properties. Conversely, for any family of fibers that satisfy $\operatorname*{ess\,sup}_{x\in\Omega} \nu_x(\Omega)<\infty$, if the operator $A$ defined by

Figures (1)

  • Figure 1: Hierarchy of DIDM spaces: the space of all DIDMs (black); its strict subset of graphon-DIDMs, isomorphic to graphon-signals (blue); and the space of dense-DIDMs, isomorphic to graph-signals, shown as a dense subset (red dots).

Theorems & Definitions (152)

  • Definition 2.1: Spaces of IDMs and DIDMs
  • Definition 2.2: The Unbalanced Wasserstein Distance
  • Definition 3.1: Graphops
  • Remark 3.2
  • Theorem 3.3
  • Definition 3.4: $k$-Profile
  • Definition 3.5
  • Definition 3.6: Action Metric
  • Theorem 4.1
  • Definition 5.1: Bofop-IDMs and Bofop-DIDMs
  • ...and 142 more