Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Aggregation-diffusion with Metastability

Kaiyuan Cui, Xinyan Wang, Zicheng Zhang, Weichen Zhao

TL;DR

This paper introduces GRADE, a Graph Neural Network framework built on aggregation-diffusion equations that balance nonlinear diffusion with interaction-driven aggregation to produce metastable, clustered node representations. By embedding the aggregation-diffusion dynamics into a Neural ODE, GRADE provides a flexible, general approach that subsumes diffusion-only GNNs and connects to classical GNNs. The key contribution is showing that metastability can mitigate over-smoothing, evidenced by sustained Dirichlet energy and competitive accuracy across both homophilic and heterophilic graphs. The work offers a principled, physics-inspired viewpoint on graph representation learning with practical benefits for robust, expressive GNNs.

Abstract

Continuous graph neural models based on differential equations have expanded the architecture of graph neural networks (GNNs). Due to the connection between graph diffusion and message passing, diffusion-based models have been widely studied. However, diffusion naturally drives the system towards an equilibrium state, leading to issues like over-smoothing. To this end, we propose GRADE inspired by graph aggregation-diffusion equations, which includes the delicate balance between nonlinear diffusion and aggregation induced by interaction potentials. The node representations obtained through aggregation-diffusion equations exhibit metastability, indicating that features can aggregate into multiple clusters. In addition, the dynamics within these clusters can persist for long time periods, offering the potential to alleviate over-smoothing effects. This nonlinear diffusion in our model generalizes existing diffusion-based models and establishes a connection with classical GNNs. We prove that GRADE achieves competitive performance across various benchmarks and alleviates the over-smoothing issue in GNNs evidenced by the enhanced Dirichlet energy.

Graph Neural Aggregation-diffusion with Metastability

TL;DR

This paper introduces GRADE, a Graph Neural Network framework built on aggregation-diffusion equations that balance nonlinear diffusion with interaction-driven aggregation to produce metastable, clustered node representations. By embedding the aggregation-diffusion dynamics into a Neural ODE, GRADE provides a flexible, general approach that subsumes diffusion-only GNNs and connects to classical GNNs. The key contribution is showing that metastability can mitigate over-smoothing, evidenced by sustained Dirichlet energy and competitive accuracy across both homophilic and heterophilic graphs. The work offers a principled, physics-inspired viewpoint on graph representation learning with practical benefits for robust, expressive GNNs.

Abstract

Continuous graph neural models based on differential equations have expanded the architecture of graph neural networks (GNNs). Due to the connection between graph diffusion and message passing, diffusion-based models have been widely studied. However, diffusion naturally drives the system towards an equilibrium state, leading to issues like over-smoothing. To this end, we propose GRADE inspired by graph aggregation-diffusion equations, which includes the delicate balance between nonlinear diffusion and aggregation induced by interaction potentials. The node representations obtained through aggregation-diffusion equations exhibit metastability, indicating that features can aggregate into multiple clusters. In addition, the dynamics within these clusters can persist for long time periods, offering the potential to alleviate over-smoothing effects. This nonlinear diffusion in our model generalizes existing diffusion-based models and establishes a connection with classical GNNs. We prove that GRADE achieves competitive performance across various benchmarks and alleviates the over-smoothing issue in GNNs evidenced by the enhanced Dirichlet energy.
Paper Structure (19 sections, 1 theorem, 30 equations, 2 figures, 2 tables)

This paper contains 19 sections, 1 theorem, 30 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background of Graph Neural Networks
  3. A Brief Review of Metastability
  4. GRADE with Metastability
  5. Aggregation-diffusion Equations on Graphs
  6. Metastability of Aggregation-diffusion Equation
  7. Our model
  8. Related work
  9. Continuous-time GNNs
  10. Classical GNNs
  11. Experiments
  12. Node Classification
  13. Datasets.
  14. Baselines
  15. Results on homophilic dataset.
  16. ...and 4 more sections

Key Result

Theorem 4.2

Suppose that the graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ is connected and there is a node $u^{*}\in\mathcal{V}$ with degree 1. Then the feature determined by Aggregation-diffusion equations GAD_eq on graph will mitigate over-smoothing.

Figures (2)

  • Figure 1: An illustration for evalution of GRADE. Due to the appearance of metastable behavior in nonlinear aggregation-diffusion equations, graph features $x(t)$ may be clustered and contained in several disconnected components. Although the metastable states will eventually transition to the ultimate equilibrium state, the system still requires a large amount of time lingering in each metastable state.
  • Figure 2: Dirichlet energy on the synthetic random graph.

Theorems & Definitions (3)

  • Definition 4.1
  • Theorem 4.2
  • proof : proof of Theorem \ref{['thm3.2']}