CODE: A global approach to ODE dynamics learning

TL;DR

The paper tackles learning ODE dynamics from sparse and noisy observations and introduces ChaosODE, a global, orthonormal polynomial chaos expansion for the RHS. By constructing a data-driven orthonormal basis and optimizing coefficients with a robust, multi-stage pipeline, ChaosODE achieves strong extrapolation to unseen initial conditions and outperforms NeuralODE and KernelODE on a Lotka–Volterra benchmark. Key contributions include a detailed comparison of RHS representations, a principled basis construction via moment matching, and practical guidelines for stable optimization, along with open-source code. The approach offers a principled inductive bias for dynamics learning, enabling reliable predictions in data-constrained scientific settings with polynomial-like dynamics.

Abstract

Ordinary differential equations (ODEs) are a conventional way to describe the observed dynamics of physical systems. Scientists typically hypothesize about dynamical behavior, propose a mathematical model, and compare its predictions to data. However, modern computing and algorithmic advances now enable purely data-driven learning of governing dynamics directly from observations. In data-driven settings, one learns the ODE's right-hand side (RHS). Dense measurements are often assumed, yet high temporal resolution is typically both cumbersome and expensive. Consequently, one usually has only sparsely sampled data. In this work we introduce ChaosODE (CODE), a Polynomial Chaos ODE Expansion in which we use an arbitrary Polynomial Chaos Expansion (aPCE) for the ODE's right-hand side, resulting in a global orthonormal polynomial representation of dynamics. We evaluate the performance of CODE in several experiments on the Lotka-Volterra system, across varying noise levels, initial conditions, and predictions far into the future, even on previously unseen initial conditions. CODE exhibits remarkable extrapolation capabilities even when evaluated under novel initial conditions and shows advantages compared to well-examined methods using neural networks (NeuralODE) or kernel approximators (KernelODE) as the RHS representer. We observe that the high flexibility of NeuralODE and KernelODE degrades extrapolation capabilities under scarce data and measurement noise. Finally, we provide practical guidelines for robust optimization of dynamics-learning problems and illustrate them in the accompanying code.

Figures (17)

• Figure 1: Numerical Lotka-Volterra ODE solution and $N=35$ training data points over time.
• Figure 2: Numerical Lotka-Volterra ODE solution and $N=35$ training data points in the state space
• Figure 3: Phase space visualization of three different modeling approaches for the Lotka-Volterra system in S1. The heatmap shows vector field magnitude, streamlines represent flow direction, and colored trajectories show model predictions from two initial conditions, resulting in two separate (orange) periodic orbits. Training data points are marked with crosses.
• Figure 4: Convergence of the MSE ex-it error as training data size increases from 10 to 100 samples. Training data contains no noise ($\sigma=0.0$). Stars indicate the best-performing model for each combination of training data size and basis function.
• Figure 5: Convergence of the MSE ex-oot error as training data size increases from 10 to 100 samples. Training data contains no noise ($\sigma=0.0$). Stars indicate the best-performing model for each combination of training data size and basis function.