Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On dimensionality of feature vectors in MPNNs

César Bravo, Alexander Kozachinskiy, Cristóbal Rojas

TL;DR

This work proves that MPNNs with one-dimensional feature vectors, coupled with any non-polynomial analytic activation, are uniformly equivalent to the WL test across graphs of any size. By constructing a 1-D MPNN with a single parameter $oldsymbol{\gamma} ext{ in }(0,1)$ and a specific update rule, the authors show that WL labels can be reconstructed and preserved through iterations, aided by a key linear-independence argument over $ obrace ext{ } obrace ext{Q}$. They provide a simple proof and substantial experimental validation, demonstrating that random choices of $oldsymbol{\gamma}$ achieve perfect WL simulation on large graph sets and that the required precision scales approximately as $O( obreak ext{log} n)$. The paper further discusses extensions to higher-order WL tests and analyzes implications for generalization and practical network design. Overall, the results indicate that low-dimensional feature spaces need not compromise the expressive power of MPNNs relative to the WL hierarchy.

Abstract

We revisit the classical result of Morris et al.~(AAAI'19) that message-passing graphs neural networks (MPNNs) are equal in their distinguishing power to the Weisfeiler--Leman (WL) isomorphism test. Morris et al.~show their simulation result with ReLU activation function and $O(n)$-dimensional feature vectors, where $n$ is the number of nodes of the graph. By introducing randomness into the architecture, Aamand et al.~(NeurIPS'22) were able to improve this bound to $O(\log n)$-dimensional feature vectors, again for ReLU activation, although at the expense of guaranteeing perfect simulation only with high probability. Recently, Amir et al.~(NeurIPS'23) have shown that for any non-polynomial analytic activation function, it is enough to use just 1-dimensional feature vectors. In this paper, we give a simple proof of the result of Amit et al.~and provide an independent experimental validation of it.

On dimensionality of feature vectors in MPNNs

TL;DR

This work proves that MPNNs with one-dimensional feature vectors, coupled with any non-polynomial analytic activation, are uniformly equivalent to the WL test across graphs of any size. By constructing a 1-D MPNN with a single parameter and a specific update rule, the authors show that WL labels can be reconstructed and preserved through iterations, aided by a key linear-independence argument over . They provide a simple proof and substantial experimental validation, demonstrating that random choices of achieve perfect WL simulation on large graph sets and that the required precision scales approximately as . The paper further discusses extensions to higher-order WL tests and analyzes implications for generalization and practical network design. Overall, the results indicate that low-dimensional feature spaces need not compromise the expressive power of MPNNs relative to the WL hierarchy.

Abstract

We revisit the classical result of Morris et al.~(AAAI'19) that message-passing graphs neural networks (MPNNs) are equal in their distinguishing power to the Weisfeiler--Leman (WL) isomorphism test. Morris et al.~show their simulation result with ReLU activation function and -dimensional feature vectors, where is the number of nodes of the graph. By introducing randomness into the architecture, Aamand et al.~(NeurIPS'22) were able to improve this bound to -dimensional feature vectors, again for ReLU activation, although at the expense of guaranteeing perfect simulation only with high probability. Recently, Amir et al.~(NeurIPS'23) have shown that for any non-polynomial analytic activation function, it is enough to use just 1-dimensional feature vectors. In this paper, we give a simple proof of the result of Amit et al.~and provide an independent experimental validation of it.
Paper Structure (11 sections, 5 theorems, 42 equations, 2 figures)

This paper contains 11 sections, 5 theorems, 42 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Key contributions.
  3. Preliminaries
  4. MPNNs.
  5. Non-polynomial analytic activation functions.
  6. Proof of the main result
  7. Experiments
  8. Uniform perfect simulation
  9. Dependence on precision bits
  10. Higher-order MPNN and WL tests
  11. Conclusion and future work

Key Result

Proposition 2.1

Let $f: \mathbb{R}\to\mathbb{R}$ be an analytic function. Suppose there is a point $x_0$ where all the derivatives of $f$ vanish. Then $f$ is the constant zero function.

Figures (2)

  • Figure 1: Number of perfect simulations of $\mathcal{M}_\gamma$ among the 300 random graphs, for 50 different values of $\gamma$ chosen uniformly at random. Lottery $\gamma$s are those that achieved perfect simulation on all the 300 graphs -- 45 in the case of Erdos-Renyi graphs, and 44 for scale-free graphs. For each case, the value of a few $\gamma$s is also shown.
  • Figure 2: The figure on top shows the number of precision bits required for perfect simulation as a function of the size of the graph. The other figures display the quality of the simulation as a function of the number of precision bits. For each precision and each value of $\gamma$, the number of classes in the final labelling output by $\mathcal{M}_ \gamma$ are potted. The true number of classes in the case of CORA is 2365, whereas for the big (5000 nodes) Erdos-Renyi graph is 3729.

Theorems & Definitions (8)

  • Proposition 2.1
  • proof
  • Theorem 3.1: Restatement of Theorem \ref{['thm_main']}
  • Lemma 3.2
  • Lemma 3.3
  • proof : Proof of Lemma \ref{['ind_lemma']}
  • Theorem 5.1
  • proof