Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND

TL;DR

FROND addresses oversmoothing and limited depth in continuous GNNs by introducing memory through the Caputo time-fractional derivative $D_t^\beta$, yielding memory-dependent graph diffusion. The framework unifies and extends multiple integer-order GNNs by replacing instantaneous updates with fractional dynamics, and it offers a non-Markovian random-walk interpretation that explains slower, algebraic convergence to stationarity. Key contributions include a generalized FROND formulation, fractional models like F-GRAND, F-CDE, and F-GREAD, a suite of solvers (predictor, predictor-corrector, short-memory, L1), and comprehensive experiments across node and graph tasks showing consistent performance gains and improved robustness. The work demonstrates that fractional dynamics can better capture dataset fractality and long-range dependencies, offering a practical and extensible pathway to more memory-aware graph learning without adding backbone parameters. Code availability further enables adoption and exploration of memory-enabled diffusion in graphs.

Abstract

We introduce the FRactional-Order graph Neural Dynamical network (FROND), a new continuous graph neural network (GNN) framework. Unlike traditional continuous GNNs that rely on integer-order differential equations, FROND employs the Caputo fractional derivative to leverage the non-local properties of fractional calculus. This approach enables the capture of long-term dependencies in feature updates, moving beyond the Markovian update mechanisms in conventional integer-order models and offering enhanced capabilities in graph representation learning. We offer an interpretation of the node feature updating process in FROND from a non-Markovian random walk perspective when the feature updating is particularly governed by a diffusion process. We demonstrate analytically that oversmoothing can be mitigated in this setting. Experimentally, we validate the FROND framework by comparing the fractional adaptations of various established integer-order continuous GNNs, demonstrating their consistently improved performance and underscoring the framework's potential as an effective extension to enhance traditional continuous GNNs. The code is available at \url{https://github.com/zknus/ICLR2024-FROND}.

Figures (5)

• Figure 1: Diagrams of fractional Adams–Bashforth–Moulton method with full (left) and short (right) memory.
• Figure 2: oversmoothing mitigation.
• Figure 3: Diagrams of fractional Adams–Bashforth–Moulton method with full (left) and short (right) memory.
• Figure 4: Model discretization in FROND with the basic predictor solver. Unlike the Euler discretization in ODEs, FDEs incorporate connections to historical times, introducing memory effects. Specifically, the dark blue connections observed in FDEs are absent in ODEs. The weight of these skip connections correlates with $\mu_{j, k}(\beta)$ as detailed in \ref{['eq.pre']}.
• Figure 5: The fractal dim of datasets. We use the Compact-Box-Burning algorithm in song2007calculate to compute the log-log slope (fractal dim) of the box size and the minimum number of boxes needed to cover the graph.

Theorems & Definitions (2)

• proof
• proof