Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems

Xin Li, Jingdong Zhang, Qunxi Zhu, Chengli Zhao, Xue Zhang, Xiaojun Duan, Wei Lin

TL;DR

This work tackles learning dynamical systems from noisy, limited time-series data where governing equations are unknown, addressing the high computational cost and local-optima susceptibility of standard Neural ODEs. It introduces Fourier NODEs (FNODEs), a simulation-free framework that uses a $K$-term Fourier expansion to estimate temporal gradients and, for PDEs, spatial gradients, training a neural network to match the gradient flow via a loss function, without requiring ODE solves. A data-augmentation loop further boosts gradient estimates and robustness, and the approach extends to PDEs with a resolution-invariant property. Empirically, FNODEs outperform baselines across parametric ODEs, parametric PDEs, and a real-world polar-motion dataset, achieving orders-of-magnitude faster training and improved predictive accuracy, including long-horizon forecasts. The method offers a scalable, robust pathway for data-driven dynamical modeling and suggests future integration with Koopman theory and uncertainty quantification to handle more complex, high-dimensional systems.

Abstract

Modeling complex systems using standard neural ordinary differential equations (NODEs) often faces some essential challenges, including high computational costs and susceptibility to local optima. To address these challenges, we propose a simulation-free framework, called Fourier NODEs (FNODEs), that effectively trains NODEs by directly matching the target vector field based on Fourier analysis. Specifically, we employ the Fourier analysis to estimate temporal and potential high-order spatial gradients from noisy observational data. We then incorporate the estimated spatial gradients as additional inputs to a neural network. Furthermore, we utilize the estimated temporal gradient as the optimization objective for the output of the neural network. Later, the trained neural network generates more data points through an ODE solver without participating in the computational graph, facilitating more accurate estimations of gradients based on Fourier analysis. These two steps form a positive feedback loop, enabling accurate dynamics modeling in our framework. Consequently, our approach outperforms state-of-the-art methods in terms of training time, dynamics prediction, and robustness. Finally, we demonstrate the superior performance of our framework using a number of representative complex systems.

From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems

TL;DR

This work tackles learning dynamical systems from noisy, limited time-series data where governing equations are unknown, addressing the high computational cost and local-optima susceptibility of standard Neural ODEs. It introduces Fourier NODEs (FNODEs), a simulation-free framework that uses a -term Fourier expansion to estimate temporal gradients and, for PDEs, spatial gradients, training a neural network to match the gradient flow via a loss function, without requiring ODE solves. A data-augmentation loop further boosts gradient estimates and robustness, and the approach extends to PDEs with a resolution-invariant property. Empirically, FNODEs outperform baselines across parametric ODEs, parametric PDEs, and a real-world polar-motion dataset, achieving orders-of-magnitude faster training and improved predictive accuracy, including long-horizon forecasts. The method offers a scalable, robust pathway for data-driven dynamical modeling and suggests future integration with Koopman theory and uncertainty quantification to handle more complex, high-dimensional systems.

Abstract

Modeling complex systems using standard neural ordinary differential equations (NODEs) often faces some essential challenges, including high computational costs and susceptibility to local optima. To address these challenges, we propose a simulation-free framework, called Fourier NODEs (FNODEs), that effectively trains NODEs by directly matching the target vector field based on Fourier analysis. Specifically, we employ the Fourier analysis to estimate temporal and potential high-order spatial gradients from noisy observational data. We then incorporate the estimated spatial gradients as additional inputs to a neural network. Furthermore, we utilize the estimated temporal gradient as the optimization objective for the output of the neural network. Later, the trained neural network generates more data points through an ODE solver without participating in the computational graph, facilitating more accurate estimations of gradients based on Fourier analysis. These two steps form a positive feedback loop, enabling accurate dynamics modeling in our framework. Consequently, our approach outperforms state-of-the-art methods in terms of training time, dynamics prediction, and robustness. Finally, we demonstrate the superior performance of our framework using a number of representative complex systems.
Paper Structure (20 sections, 1 theorem, 33 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 20 sections, 1 theorem, 33 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Framework of FNODEs
  4. Illustrative Experiments
  5. Parametric ODEs
  6. Parametric PDEs
  7. Additional Validation and Discussion of Our Framework
  8. Handling Sparse and Irregularly Sampled Time Series.
  9. Resolution-invariant property of the FNODEs.
  10. Experimental results on a real-world system
  11. Concluding Remarks
  12. Theorems and Algorithms
  13. Proof of Theorem 1
  14. Execution process of the FNODEs framework
  15. Experimental Hyperparameters
  16. ...and 5 more sections

Key Result

Theorem 1

Assume $\bm{h}(t)$ is a $T$-periodic and thrice differentiable function, and its second derivative $\bm{h}"(t)$ satisfies the Lipschitz condition on $[0,T]$. Then, for any $\epsilon > 0$, there exists a positive integer $K_0$ such that for any $K > K_0$ and for all $t\in [0,T]$, we have: where $\bm{H}_K'(t) = \frac{2\pi}{T} \sum_{k=1}^K k[\bm{a}_k\sin(2\pi kt/T)+\bm{b}_k\cos(2\pi kt/T)]$. This im

Figures (9)

  • Figure 1: Illustration of the FNODEs framework.
  • Figure 2: The experimental results of system (\ref{['E_2Dode']}) using FNODEs. (a) Estimation of the derivatives based on the observational data using Fourier analysis. (b) The test prediction using the trained model. (c) Comparison of the FNODEs and baseline methods in terms of training time and prediction error.
  • Figure 3: The experimental results of the KDV system using FNODEs Method. (a) The variation of the training and validation loss during the training process. (b) Plot of $u_1(t)$ from a test data. (c) These three subplots from top to bottom represent the ground truth, predicted results, and prediction errors of test data.
  • Figure 4: The experimental results of the NS system using FNODEs Method. (a) The initial state $s_0$ of a certain test data. (b) The initial state $u_0$ of the parameter $u(x,y,0)$. (c)-(e) These subfigures depict the ground truth, prediction results, and prediction errors at different time instances. From left to right, the time instances $t$ correspond to 2.5s, 5s, 7.5s, and 10s.
  • Figure 5: Experimental results of the data augmentation and the resolution-invariant predictions using the FNODEs. In system (\ref{['E_2Dode']}), (a) Prediction errors of time gradients under different sampling rates $N$ and cutoff frequencies $K$. (b) A boxplot depicting the testing errors at different stages under the utilization of feedback data, with the median of prediction errors represented by a yellow horizontal line. (c)-(e) The prediction results of test data under different feedback settings (0, 200, and 600 feedback points respectively). In DR system, (f) The prediction MSE of temporal gradient under different $N_x$ in training data. (g) A training sample with a low resolution. (h) A predicted sample with a high resolution.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 1