Neural Variable-Order Fractional Differential Equation Networks
Wenjun Cui, Qiyu Kang, Xuhao Li, Kai Zhao, Wee Peng Tay, Weihua Deng, Yidong Li
TL;DR
The paper tackles memory-driven dynamics in neural models by introducing Neural Variable-Order FDEs (NvoFDE), where the derivative order is a learnable function alpha(t, x(t)) that adapts during computation. It unifies neural ODEs, constant-order neural FDEs, and variable-order formulations within a single framework, and extends this to graph neural networks via V-FROND variants. The authors propose practical solvers (L1 and ABM) and show through extensive experiments on VP equations, homophilic/heterophilic graphs, and image data that learnable variable orders improve predictive performance and memory modeling over fixed-order baselines. The work highlights the potential for adaptive memory in physics-informed learning, graph representation, and broader dynamical systems applications, with code available for reproduction.
Abstract
Neural differential equation models have garnered significant attention in recent years for their effectiveness in machine learning applications.Among these, fractional differential equations (FDEs) have emerged as a promising tool due to their ability to capture memory-dependent dynamics, which are often challenging to model with traditional integer-order approaches.While existing models have primarily focused on constant-order fractional derivatives, variable-order fractional operators offer a more flexible and expressive framework for modeling complex memory patterns. In this work, we introduce the Neural Variable-Order Fractional Differential Equation network (NvoFDE), a novel neural network framework that integrates variable-order fractional derivatives with learnable neural networks.Our framework allows for the modeling of adaptive derivative orders dependent on hidden features, capturing more complex feature-updating dynamics and providing enhanced flexibility. We conduct extensive experiments across multiple graph datasets to validate the effectiveness of our approach.Our results demonstrate that NvoFDE outperforms traditional constant-order fractional and integer models across a range of tasks, showcasing its superior adaptability and performance.