Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Reaction Diffusion Models

Moshe Eliasof, Eldad Haber, Eran Treister

TL;DR

This work introduces RDGNN, a novel graph neural network family derived from discretizing reaction-diffusion equations on graphs and inspired by Turing instabilities. It learns both the diffusion coefficients and a learnable reaction term, employing an implicit-explicit time integration scheme to balance stability and expressivity, enabling modeling of both smooth diffusion and non-smooth pattern formation on heterogeneous and spatio-temporal graphs. The paper provides theoretical analysis of local stability, details the reaction and diffusion function parameterizations, and demonstrates superior or competitive performance on homophilic, heterophilic, and spatio-temporal datasets, along with thorough ablations. Overall, RDGNN expands GNN design by integrating dynamic, trainable RD processes that capture complex patterns beyond standard diffusion-based models, with practical impact on diverse graph-based learning tasks.

Abstract

The integration of Graph Neural Networks (GNNs) and Neural Ordinary and Partial Differential Equations has been extensively studied in recent years. GNN architectures powered by neural differential equations allow us to reason about their behavior, and develop GNNs with desired properties such as controlled smoothing or energy conservation. In this paper we take inspiration from Turing instabilities in a Reaction Diffusion (RD) system of partial differential equations, and propose a novel family of GNNs based on neural RD systems. We \textcolor{black}{demonstrate} that our RDGNN is powerful for the modeling of various data types, from homophilic, to heterophilic, and spatio-temporal datasets. We discuss the theoretical properties of our RDGNN, its implementation, and show that it improves or offers competitive performance to state-of-the-art methods.

Graph Neural Reaction Diffusion Models

TL;DR

This work introduces RDGNN, a novel graph neural network family derived from discretizing reaction-diffusion equations on graphs and inspired by Turing instabilities. It learns both the diffusion coefficients and a learnable reaction term, employing an implicit-explicit time integration scheme to balance stability and expressivity, enabling modeling of both smooth diffusion and non-smooth pattern formation on heterogeneous and spatio-temporal graphs. The paper provides theoretical analysis of local stability, details the reaction and diffusion function parameterizations, and demonstrates superior or competitive performance on homophilic, heterophilic, and spatio-temporal datasets, along with thorough ablations. Overall, RDGNN expands GNN design by integrating dynamic, trainable RD processes that capture complex patterns beyond standard diffusion-based models, with practical impact on diverse graph-based learning tasks.

Abstract

The integration of Graph Neural Networks (GNNs) and Neural Ordinary and Partial Differential Equations has been extensively studied in recent years. GNN architectures powered by neural differential equations allow us to reason about their behavior, and develop GNNs with desired properties such as controlled smoothing or energy conservation. In this paper we take inspiration from Turing instabilities in a Reaction Diffusion (RD) system of partial differential equations, and propose a novel family of GNNs based on neural RD systems. We \textcolor{black}{demonstrate} that our RDGNN is powerful for the modeling of various data types, from homophilic, to heterophilic, and spatio-temporal datasets. We discuss the theoretical properties of our RDGNN, its implementation, and show that it improves or offers competitive performance to state-of-the-art methods.
Paper Structure (34 sections, 1 theorem, 25 equations, 4 figures, 12 tables)

This paper contains 34 sections, 1 theorem, 25 equations, 4 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Neural Networks as Diffusion Processes
  4. Neural ODEs and PDEs
  5. Reaction Diffusion
  6. Neural Graph Reaction Diffusion
  7. Reaction Diffusion Systems
  8. Reaction Diffusion on Graphs
  9. Time Integration
  10. Explicit Integration
  11. Implicit-Explicit (IMEX) Integration
  12. Local Stability Analysis
  13. The Reaction and Diffusion Functions
  14. The Reaction Function
  15. The Diffusion Term
  16. ...and 19 more sections

Key Result

Theorem 3.1

Consider the discrete RD equation defined by Equation eq:discDRa, where $\hat{{\bf L}}$ is the symmetrically normalized graph Laplacian, ${\bf \Sigma} = {\rm diag}(\kappa_1, \ldots, \kappa_c)$ with $\kappa_j\textcolor{black}{\geq}0$ are the diffusion coefficients, and $f({\bf U},\theta)$ is a non-li where ${\bf J}=\nabla_{\bf U} f$, contains positive eigenvalues, then the RD system eq:discDRa is u

Figures (4)

  • Figure 1: The layer dynamics (portrayed along the x-axis) induced by our RDGNN vs. existing parabolic and hyperbolic dynamics.
  • Figure 1: An example of the ground-truth labels of (a) Cornell (b) Wisconsin, and (c) Texas. Different colors indicate different labels. It is evident that non-smooth patterns exist in the data, which are suitable to be modeled with RD systems.
  • Figure 2: The accuracy (%) on homophilic and heterophilic datasets vs. model depth. Our RDGNN models do not suffer from oversmoothing.
  • Figure 3: RDGNN alleviates oversmoothing regardless of the inclusion of ${\bf U}_0$ in the reaction term. The base RDGNN here is RDGNN-I.

Theorems & Definitions (3)

  • Theorem 3.1
  • Proof 1
  • Remark 3.2