Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using Random Noise Equivariantly to Boost Graph Neural Networks Universally

Xiyuan Wang, Muhan Zhang

TL;DR

This work addresses the challenge of enhancing Graph Neural Networks (GNNs) expressivity without sacrificing generalization when using random noise as input features. It introduces Equivariant Noise GNN (ENGNN), a symmetry-aware architecture that processes noisy inputs in an invariant-equivariant fashion via a novel DeepSet-inspired aggregator, preserving permutation symmetries on both nodes and noise channels. The authors provide a theoretical framework linking noise-induced sample complexity to noise-space symmetry and prove that the proposed aggregator can universally approximate equivariant functions, while experiments show ENGNN improves performance across node, link, subgraph, and graph-level tasks with comparable scalability to standard MPNNs. Overall, ENGNN offers a general, task-agnostic method to boost GNN expressivity across diverse graph problems, with strong theoretical guarantees and broad empirical validation.

Abstract

Recent advances in Graph Neural Networks (GNNs) have explored the potential of random noise as an input feature to enhance expressivity across diverse tasks. However, naively incorporating noise can degrade performance, while architectures tailored to exploit noise for specific tasks excel yet lack broad applicability. This paper tackles these issues by laying down a theoretical framework that elucidates the increased sample complexity when introducing random noise into GNNs without careful design. We further propose Equivariant Noise GNN (ENGNN), a novel architecture that harnesses the symmetrical properties of noise to mitigate sample complexity and bolster generalization. Our experiments demonstrate that using noise equivariantly significantly enhances performance on node-level, link-level, subgraph, and graph-level tasks and achieves comparable performance to models designed for specific tasks, thereby offering a general method to boost expressivity across various graph tasks.

Using Random Noise Equivariantly to Boost Graph Neural Networks Universally

TL;DR

This work addresses the challenge of enhancing Graph Neural Networks (GNNs) expressivity without sacrificing generalization when using random noise as input features. It introduces Equivariant Noise GNN (ENGNN), a symmetry-aware architecture that processes noisy inputs in an invariant-equivariant fashion via a novel DeepSet-inspired aggregator, preserving permutation symmetries on both nodes and noise channels. The authors provide a theoretical framework linking noise-induced sample complexity to noise-space symmetry and prove that the proposed aggregator can universally approximate equivariant functions, while experiments show ENGNN improves performance across node, link, subgraph, and graph-level tasks with comparable scalability to standard MPNNs. Overall, ENGNN offers a general, task-agnostic method to boost GNN expressivity across diverse graph problems, with strong theoretical guarantees and broad empirical validation.

Abstract

Recent advances in Graph Neural Networks (GNNs) have explored the potential of random noise as an input feature to enhance expressivity across diverse tasks. However, naively incorporating noise can degrade performance, while architectures tailored to exploit noise for specific tasks excel yet lack broad applicability. This paper tackles these issues by laying down a theoretical framework that elucidates the increased sample complexity when introducing random noise into GNNs without careful design. We further propose Equivariant Noise GNN (ENGNN), a novel architecture that harnesses the symmetrical properties of noise to mitigate sample complexity and bolster generalization. Our experiments demonstrate that using noise equivariantly significantly enhances performance on node-level, link-level, subgraph, and graph-level tasks and achieves comparable performance to models designed for specific tasks, thereby offering a general method to boost expressivity across various graph tasks.

Paper Structure

This paper contains 32 sections, 7 theorems, 33 equations, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Message Passing Neural Network (MPNN) MPNN
  4. Equivariance and Invariance
  5. Generalization
  6. Notations
  7. Symmetry in Noise Space
  8. Sample Complexity for Models with Graph and Noise Inputs
  9. Reducing Sample Complexity Through Symmetry
  10. Equivariant Noise Graph Neural Network
  11. Equivariant Aggregator
  12. Architecture
  13. Message-Passing Layer
  14. Pooling Layer
  15. Expressivity
  16. ...and 17 more sections

Key Result

Theorem 3.1

Let $T \subseteq \{ t: \mathcal{Z} \to \mathcal{Z} \}$. Assume that all hypotheses $h \in H$ are $C_G$-Lipschitz with respect to the graph space $\mathcal{G}$ and $C_Z$-Lipschitz with respect to the noise space $\mathcal{Z}$, and that $h$ is $T$-invariant. If the algorithm is also $T$-invariant, the

Figures (1)

  • Figure 1: Example of GNNs with noise as auxiliary input. Each node has $C$ channel noise.

Theorems & Definitions (10)

  • Theorem 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 4.1
  • Proposition 4.2
  • Theorem 4.3
  • Theorem 4.4
  • proof
  • proof
  • proof