Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs

Matthieu Cordonnier, Nicolas Keriven, Nicolas Tremblay, Samuel Vaiter

TL;DR

The paper studies convergence of multilayer MPGNNs with generic aggregation on large random graphs to continuous GNNs (cMPGNNs). It defines a continuous counterpart on a latent space with a connectivity kernel and analyzes when the discrete MPGNN outputs concentrate near the continuous limit as $n$ grows, providing non-asymptotic, high-probability bounds. Key contributions include a McDiarmid-based bound for Lipschitz-type aggregations and a separate, dimension-dependent bound for max aggregation, along with detailed verification across several common aggregation schemes (mean, degree-normalized, attention, generalized mean, and max). The results extend prior convergence analyses from SGNNs and degree-normalized mean MPGNNs to a broad class of aggregations, informing theoretical understanding and practical behavior of GNNs on large graphs. Experimental illustrations corroborate the predicted rates, highlighting how convergence scales with latent-space dimension for mean versus max aggregations, and suggesting practical implications for theory-guided design of GNN architectures on large networks.

Abstract

We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity. Until now, this convergence was only known for architectures with aggregation functions in the form of normalized means, or, equivalently, of an application of classical operators like the adjacency matrix or the graph Laplacian. We extend such results to a large class of aggregation functions, that encompasses all classically used message passing graph neural networks, such as attention-based message passing, max convolutional message passing, (degree-normalized) convolutional message passing, or moment-based aggregation message passing. Under mild assumptions, we give non-asymptotic bounds with high probability to quantify this convergence. Our main result is based on the McDiarmid inequality. Interestingly, this result does not apply to the case where the aggregation is a coordinate-wise maximum. We treat this case separately and obtain a different convergence rate.

Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs

TL;DR

The paper studies convergence of multilayer MPGNNs with generic aggregation on large random graphs to continuous GNNs (cMPGNNs). It defines a continuous counterpart on a latent space with a connectivity kernel and analyzes when the discrete MPGNN outputs concentrate near the continuous limit as grows, providing non-asymptotic, high-probability bounds. Key contributions include a McDiarmid-based bound for Lipschitz-type aggregations and a separate, dimension-dependent bound for max aggregation, along with detailed verification across several common aggregation schemes (mean, degree-normalized, attention, generalized mean, and max). The results extend prior convergence analyses from SGNNs and degree-normalized mean MPGNNs to a broad class of aggregations, informing theoretical understanding and practical behavior of GNNs on large graphs. Experimental illustrations corroborate the predicted rates, highlighting how convergence scales with latent-space dimension for mean versus max aggregations, and suggesting practical implications for theory-guided design of GNN architectures on large networks.

Abstract

We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity. Until now, this convergence was only known for architectures with aggregation functions in the form of normalized means, or, equivalently, of an application of classical operators like the adjacency matrix or the graph Laplacian. We extend such results to a large class of aggregation functions, that encompasses all classically used message passing graph neural networks, such as attention-based message passing, max convolutional message passing, (degree-normalized) convolutional message passing, or moment-based aggregation message passing. Under mild assumptions, we give non-asymptotic bounds with high probability to quantify this convergence. Our main result is based on the McDiarmid inequality. Interestingly, this result does not apply to the case where the aggregation is a coordinate-wise maximum. We treat this case separately and obtain a different convergence rate.
Paper Structure (48 sections, 20 theorems, 144 equations, 1 figure, 1 table)

This paper contains 48 sections, 20 theorems, 144 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related work.
  4. Outline.
  5. Notations and Definitions
  6. Graph-related Definitions
  7. Graph.
  8. Graph signal.
  9. Graph isomorphism.
  10. Random Graph Models
  11. Random graph model and random graphs.
  12. Random graph model isomorphism.
  13. Message-Passing Graph Neural Networks
  14. Continuous Message-Passing GNNs on Random Graph Models
  15. Continuous Message-Passing GNNs as Limits of Discrete Message-Passing GNNs on Large Random Graphs
  16. ...and 33 more sections

Key Result

Proposition 1

With the definition of the message-passing from eq: general message-passingeq: mpgnn output, $\Theta$ and $\overline{\Theta}$ are respectively $\mathfrak{S}_n$-equivariant and $\mathfrak{S}_n$-invariant, in the sense that for all $\sigma \in \mathfrak{S}_n$, for all $Z\in\mathds{R}^{n\times d_0}$, w

Figures (1)

  • Figure 1: Numerical experiences for observing the trends of the rates of convergence arising from \ref{['th: main result', 'th: main result max']}. The plots are in logarithmic scale. Left: max aggregation. Right: mean aggregation. For both figures, the dashed lines represent the experimental error as the graph size increases, while the full lines represent the theoretical rates arising from \ref{['th: main result', 'th: main result max']}. This experiment has been conducted for various values of the latent space dimension $d$. The theoretical rates are $1/\sqrt{n}$ for a mean aggregation and $1/n^{1/d}$ for a max aggregation.

Theorems & Definitions (35)

  • Proposition 1: Invariance and equivariance of MPGNNs
  • Example 1: Convolutional Message-Passing
  • Example 2: Degree normalized convolution
  • Example 3: Attention based Message-Passing
  • Example 4: Generalized mean
  • Example 5: Max Convolutional Message-Passing
  • Proposition 3: Invariance and equivariance of cMPGNNs
  • Example a: Convolutional Message-Passing
  • Example b: Degree Normalized Convolutional Message-Passing
  • Example c: Attention based Message-Passing
  • ...and 25 more