Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the approximation capability of GNNs in node classification/regression tasks

Giuseppe Alessio D'Inverno, Monica Bianchini, Maria Lucia Sampoli, Franco Scarselli

TL;DR

This work tackles the limitations of prior universal-approximation results for GNNs by focusing on node-focused tasks and providing a constructive, probability-based universality framework. It shows that GNNs can approximate any measurable function that preserves the $1$-WL equivalence, with a tight depth bound of $2r-1$ iterations (where $r$ is the maximum graph size), and furnishes a method to encode unfolding trees into node features to guide architecture design. A formal equivalence is established between unfolding trees and WL color refinement, enabling a unified view of GNN expressiveness for nodes and graphs, and the results extend to GNNs with universal components, yielding density-type guarantees. Experimental validation using Graph Isomorphism Networks on QM9-derived datasets corroborates the theory, demonstrating high-accuracy approximation of WL-based targets and illustrating the practical relevance for node-level reasoning.

Abstract

Graph Neural Networks (GNNs) are a broad class of connectionist models for graph processing. Recent studies have shown that GNNs can approximate any function on graphs, modulo the equivalence relation on graphs defined by the Weisfeiler--Lehman (WL) test. However, these results suffer from some limitations, both because they were derived using the Stone--Weierstrass theorem -- which is existential in nature, -- and because they assume that the target function to be approximated must be continuous. Furthermore, all current results are dedicated to graph classification/regression tasks, where the GNN must produce a single output for the whole graph, while also node classification/regression problems, in which an output is returned for each node, are very common. In this paper, we propose an alternative way to demonstrate the approximation capability of GNNs that overcomes these limitations. Indeed, we show that GNNs are universal approximators in probability for node classification/regression tasks, as they can approximate any measurable function that satisfies the 1--WL equivalence on nodes. The proposed theoretical framework allows the approximation of generic discontinuous target functions and also suggests the GNN architecture that can reach a desired approximation. In addition, we provide a bound on the number of the GNN layers required to achieve the desired degree of approximation, namely $2r-1$, where $r$ is the maximum number of nodes for the graphs in the domain.

On the approximation capability of GNNs in node classification/regression tasks

TL;DR

This work tackles the limitations of prior universal-approximation results for GNNs by focusing on node-focused tasks and providing a constructive, probability-based universality framework. It shows that GNNs can approximate any measurable function that preserves the -WL equivalence, with a tight depth bound of iterations (where is the maximum graph size), and furnishes a method to encode unfolding trees into node features to guide architecture design. A formal equivalence is established between unfolding trees and WL color refinement, enabling a unified view of GNN expressiveness for nodes and graphs, and the results extend to GNNs with universal components, yielding density-type guarantees. Experimental validation using Graph Isomorphism Networks on QM9-derived datasets corroborates the theory, demonstrating high-accuracy approximation of WL-based targets and illustrating the practical relevance for node-level reasoning.

Abstract

Graph Neural Networks (GNNs) are a broad class of connectionist models for graph processing. Recent studies have shown that GNNs can approximate any function on graphs, modulo the equivalence relation on graphs defined by the Weisfeiler--Lehman (WL) test. However, these results suffer from some limitations, both because they were derived using the Stone--Weierstrass theorem -- which is existential in nature, -- and because they assume that the target function to be approximated must be continuous. Furthermore, all current results are dedicated to graph classification/regression tasks, where the GNN must produce a single output for the whole graph, while also node classification/regression problems, in which an output is returned for each node, are very common. In this paper, we propose an alternative way to demonstrate the approximation capability of GNNs that overcomes these limitations. Indeed, we show that GNNs are universal approximators in probability for node classification/regression tasks, as they can approximate any measurable function that satisfies the 1--WL equivalence on nodes. The proposed theoretical framework allows the approximation of generic discontinuous target functions and also suggests the GNN architecture that can reach a desired approximation. In addition, we provide a bound on the number of the GNN layers required to achieve the desired degree of approximation, namely , where is the maximum number of nodes for the graphs in the domain.

Paper Structure

This paper contains 16 sections, 13 theorems, 34 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Graphs
  5. Graph neural networks
  6. Unfolding trees and unfolding equivalence
  7. The color refinement algorithm and the Weisfeiler--Lehman test
  8. Main Results
  9. Unfolding and Weisfeiler--Lehman equivalence
  10. Approximation capability
  11. Experimental Validation
  12. Dataset
  13. Experimental setup
  14. Results
  15. Conclusion
  16. ...and 1 more sections

Key Result

Theorem 4.1.1

Let ${\mathbf G}=({\mathbf V},{\mathbf E})$ be a labeled graph. Then, for each $u,v \in {\mathbf V}$, $u \backsim_{ue} v$ holds if and only if $u \backsim_{WL} v$ holds. Moreover, for each integer $t\geq 0$, $u \backsim_{ue_t} v$ if and only if $u \backsim_{WL_t} v$. ∎

Figures (6)

  • Figure 1: An example of a graph with some unfolding trees. The symbols outside the nodes represent features. The two nodes on the left part of the graph are equivalent and have equivalent unfolding trees.
  • Figure 2: A graphical representation of the relationship between the color refinement and the unfolding equivalence, applied on nodes 1 and 4 of the given graph.
  • Figure 3: (a) A regular graph where all nodes have the same features. All unfolding trees are equal. (b) The equivalence classes when only one node has different features. (c) The equivalence classes when all nodes has different features.
  • Figure 4: In (a) and (b), two graphs $\mathbf{G}$, $\mathbf{H}$ are depicted that satisfy the lower bound of point 2 of of Theorem \ref{['th:treeDepth']}. We assume that all the nodes have the same attributes even if they are displayed with different symbols in terms of their "role" in the coloring scheme. Graphs in (a) and (b) are constructed by aggregating in a sequence two copies of the same subgraph (c); then, module (d) is added at the top of graph (a), while module (e) is added at the top of graph (b). It is worth noting that (a) and (b) do not satisfy the relation $2r-16\sqrt{r}>r$; nevertheless, adding multiple times module (c) to the tail of both (a) and (b), we can find two graphs satisfying the requested relation.
  • Figure 5: Training accuracy on subsampled QM9 datasets, increasing number of WL colors (a), and increasing hidden layer size (b). The solid line represents the average over 15 runs, the shaded area represents the confidence interval.
  • ...and 1 more figures

Theorems & Definitions (27)

  • Definition 3.3.1
  • Definition 3.3.2
  • Definition 3.4.1: WL--equivalence
  • Theorem 4.1.1
  • Theorem 4.1.2
  • Theorem 4.1.3
  • Definition 4.1.4
  • Theorem 4.1.5: Functions of unfolding trees
  • Theorem 4.2.1: Approximation by GNNs
  • Definition 4.2.2
  • ...and 17 more