Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study

Qiyu Kang, Kai Zhao, Yang Song, Yihang Xie, Yanan Zhao, Sijie Wang, Rui She, Wee Peng Tay

TL;DR

A theoretical foundation is established outlining the robustness characteristics of graph neural FDE models, highlighting that they maintain more stringent output perturbation bounds in the face of input and graph topology disturbances, compared to their integer-order counterparts.

Abstract

In this work, we rigorously investigate the robustness of graph neural fractional-order differential equation (FDE) models. This framework extends beyond traditional graph neural (integer-order) ordinary differential equation (ODE) models by implementing the time-fractional Caputo derivative. Utilizing fractional calculus allows our model to consider long-term memory during the feature updating process, diverging from the memoryless Markovian updates seen in traditional graph neural ODE models. The superiority of graph neural FDE models over graph neural ODE models has been established in environments free from attacks or perturbations. While traditional graph neural ODE models have been verified to possess a degree of stability and resilience in the presence of adversarial attacks in existing literature, the robustness of graph neural FDE models, especially under adversarial conditions, remains largely unexplored. This paper undertakes a detailed assessment of the robustness of graph neural FDE models. We establish a theoretical foundation outlining the robustness characteristics of graph neural FDE models, highlighting that they maintain more stringent output perturbation bounds in the face of input and graph topology disturbances, compared to their integer-order counterparts. Our empirical evaluations further confirm the enhanced robustness of graph neural FDE models, highlighting their potential in adversarially robust applications.

Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study

TL;DR

A theoretical foundation is established outlining the robustness characteristics of graph neural FDE models, highlighting that they maintain more stringent output perturbation bounds in the face of input and graph topology disturbances, compared to their integer-order counterparts.

Abstract

In this work, we rigorously investigate the robustness of graph neural fractional-order differential equation (FDE) models. This framework extends beyond traditional graph neural (integer-order) ordinary differential equation (ODE) models by implementing the time-fractional Caputo derivative. Utilizing fractional calculus allows our model to consider long-term memory during the feature updating process, diverging from the memoryless Markovian updates seen in traditional graph neural ODE models. The superiority of graph neural FDE models over graph neural ODE models has been established in environments free from attacks or perturbations. While traditional graph neural ODE models have been verified to possess a degree of stability and resilience in the presence of adversarial attacks in existing literature, the robustness of graph neural FDE models, especially under adversarial conditions, remains largely unexplored. This paper undertakes a detailed assessment of the robustness of graph neural FDE models. We establish a theoretical foundation outlining the robustness characteristics of graph neural FDE models, highlighting that they maintain more stringent output perturbation bounds in the face of input and graph topology disturbances, compared to their integer-order counterparts. Our empirical evaluations further confirm the enhanced robustness of graph neural FDE models, highlighting their potential in adversarially robust applications.
Paper Structure (32 sections, 3 theorems, 40 equations, 5 figures, 8 tables)

This paper contains 32 sections, 3 theorems, 40 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Neural ODE Models
  4. Adversarial Attacks and Defenses on Graphs
  5. Preliminaries
  6. Notation
  7. Graph Neural ODE Models
  8. Fractional-Order Differential Equation
  9. Numerical Solvers for FDEs
  10. Methodology
  11. FROND: Graph Neural FDE Framework
  12. Robustness of FROND under Perturbation
  13. Algorithms
  14. Experiments
  15. GMA
  16. ...and 17 more sections

Key Result

Theorem 1

diethelm2010frationalde Let $\mathbf{X}(t)$ be the solution of the initial value problem eq.frac_gra_difaaa, and let $\Tilde{\mathbf{X}}(t)$ be the solution of the initial value problem where $\varepsilon := \|\mathbf{X}_{0} - \Tilde{\mathbf{X}}_{0}\|$. Then, if $\varepsilon$ is sufficiently small, there exists some $h > 0$ such that both the functions $\mathbf{X}$ and $\Tilde{\mathbf{X}}$ are de

Figures (5)

  • Figure 1: Model discretization in FROND. Unlike the Euler discretization in graph neural ODE models, FROND incorporates connections to historical times, introducing memory effects. Specifically, the dark blue connections observed in FROND at $\beta<1$ are absent in ODEs (corresponding to $\beta=1$). The weight of these skip connections correlates with $b_{j,k+1}(\beta)$ as detailed in \ref{['eq.bjk']}.
  • Figure 2: Plot of the Mittag-Leffler function $E_{\beta}(L T^{\beta})$ against $\beta$ with $T=10$. Distinctively, for varying $L$, it displays monotonic increase over interval $[\epsilon,1]$.
  • Figure 3: The impact of $\beta$ on the robust test accuracy.
  • Figure 4: Contour $\gamma(1, \varphi)$
  • Figure 5:

Theorems & Definitions (9)

  • Remark 1
  • Definition 1: Mittag-Leffler function
  • Theorem 1
  • Remark 2
  • Theorem 2
  • Remark 3
  • Theorem 3
  • proof
  • Remark 4