Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DuSEGO: Dual Second-order Equivariant Graph Ordinary Differential Equation

Yingxu Wang, Nan Yin, Mingyan Xiao, Xinhao Yi, Siwei Liu, Shangsong Liang

TL;DR

DuSEGO addresses limitations of equivariant GNNs by introducing dual second-order graph ODEs that evolve both node coordinates and features, preserving equivariance while mitigating over-smoothing and gradient instability. The method plugs in any equivariant backbone (EGNN or SEGNN) and solves coupled second-order dynamics for $\mathbf{X}$ and $\mathbf{H}$, yielding richer temporal trajectories than traditional multi-layer GNNs. Theoretical guarantees include equivariance preservation and gradient-stability properties, while empirical results across dynamic-system modeling, graph autoencoding, and molecular property prediction show improved accuracy, data efficiency, and robustness to depth. These findings suggest DuSEGO as a flexible, physics-inspired framework for high-order graph dynamics with broad applicability in complex physical and molecular domains.

Abstract

Graph Neural Networks (GNNs) with equivariant properties have achieved significant success in modeling complex dynamic systems and molecular properties. However, their expressiveness ability is limited by: (1) Existing methods often overlook the over-smoothing issue caused by traditional GNN models, as well as the gradient explosion or vanishing problems in deep GNNs. (2) Most models operate on first-order information, neglecting that the real world often consists of second-order systems, which further limits the model's representation capabilities. To address these issues, we propose the \textbf{Du}al \textbf{S}econd-order \textbf{E}quivariant \textbf{G}raph \textbf{O}rdinary Differential Equation (\method{}) for equivariant representation. Specifically, \method{} apply the dual second-order equivariant graph ordinary differential equations (Graph ODEs) on graph embeddings and node coordinates, simultaneously. Theoretically, we first prove that \method{} maintains the equivariant property. Furthermore, we provide theoretical insights showing that \method{} effectively alleviates the over-smoothing problem in both feature representation and coordinate update. Additionally, we demonstrate that the proposed \method{} mitigates the exploding and vanishing gradients problem, facilitating the training of deep multi-layer GNNs. Extensive experiments on benchmark datasets validate the superiority of the proposed \method{} compared to baselines.

DuSEGO: Dual Second-order Equivariant Graph Ordinary Differential Equation

TL;DR

DuSEGO addresses limitations of equivariant GNNs by introducing dual second-order graph ODEs that evolve both node coordinates and features, preserving equivariance while mitigating over-smoothing and gradient instability. The method plugs in any equivariant backbone (EGNN or SEGNN) and solves coupled second-order dynamics for and , yielding richer temporal trajectories than traditional multi-layer GNNs. Theoretical guarantees include equivariance preservation and gradient-stability properties, while empirical results across dynamic-system modeling, graph autoencoding, and molecular property prediction show improved accuracy, data efficiency, and robustness to depth. These findings suggest DuSEGO as a flexible, physics-inspired framework for high-order graph dynamics with broad applicability in complex physical and molecular domains.

Abstract

Graph Neural Networks (GNNs) with equivariant properties have achieved significant success in modeling complex dynamic systems and molecular properties. However, their expressiveness ability is limited by: (1) Existing methods often overlook the over-smoothing issue caused by traditional GNN models, as well as the gradient explosion or vanishing problems in deep GNNs. (2) Most models operate on first-order information, neglecting that the real world often consists of second-order systems, which further limits the model's representation capabilities. To address these issues, we propose the \textbf{Du}al \textbf{S}econd-order \textbf{E}quivariant \textbf{G}raph \textbf{O}rdinary Differential Equation (\method{}) for equivariant representation. Specifically, \method{} apply the dual second-order equivariant graph ordinary differential equations (Graph ODEs) on graph embeddings and node coordinates, simultaneously. Theoretically, we first prove that \method{} maintains the equivariant property. Furthermore, we provide theoretical insights showing that \method{} effectively alleviates the over-smoothing problem in both feature representation and coordinate update. Additionally, we demonstrate that the proposed \method{} mitigates the exploding and vanishing gradients problem, facilitating the training of deep multi-layer GNNs. Extensive experiments on benchmark datasets validate the superiority of the proposed \method{} compared to baselines.

Paper Structure

This paper contains 28 sections, 2 theorems, 25 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Method
  5. Second-order Graph ODE
  6. DuSEGO
  7. Time Complexity Analysis
  8. DuSEGO Properties Analysis
  9. Experiments
  10. Modeling a Dynamic System
  11. Dataset
  12. Baselines.
  13. Experimental Setting.
  14. Implementation details.
  15. Results.
  16. ...and 13 more sections

Key Result

Proposition 1

Suppose the backbone Equivariant GNN $F_\theta$ of DuSEGO is E(3) equivariant and translation- invariant, the output $\mathbf{H}$ and $\mathbf{X}$ is E(3)-equivariant.

Figures (4)

  • Figure 1: Dirichlet energy $\mathbf{E}\left(\mathbf{H}^n\right)$ of node features $\mathbf{H}^n$ and $\mathbf{E}\left(\mathbf{X}^n\right)$ of node coordinates $\mathbf{X}^n$ propagated through a EGNN satorras2021n, SEGNN brandstetter2021geometric and GMN huang2021equivariant on a N-body system dataset, where we give the definition of the Dirichlet energy in Eq. \ref{['energy']}.
  • Figure 2: Visualization of ground truths (Blue) and prediction results (Red) for EGNN, SEGNN, GMN, SEGNO, DuSEGO-EGNN, and DuSEGO-SEGNN under 1000 timesteps on the N-body system dataset.
  • Figure 3: Data efficiency comparisons among EGNN, SEGNN, GMN, GraphCon, SEGNO, DuSEGO-EGNN , and DuSEGO-SEGNN on the N-body dataset.
  • Figure 4: Dirichlet energy $\mathbf{E}\left(\mathbf{H}^n\right)$ of node features $\mathbf{H}^n$ and $\mathbf{E}\left(\mathbf{X}^n\right)$ of node coordinates $\mathbf{X}^n$ propagated through a EGNN and DuSEGO-EGNN for two different values of $\alpha=0,1$ and $\gamma_1 = \gamma_2=1$ in Eq. \ref{['ode1']} and Eq. \ref{['ode2']}.

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2