Grammar-based Ordinary Differential Equation Discovery

TL;DR

The paper tackles the ill-posed challenge of discovering ODEs from noisy, sparse data by introducing GODE, a grammar-based framework that restricts the symbolic search space with formal grammars, embeds candidate expressions in a latent space via a Grammar Variational Autoencoder, and performs stochastic latent-space search with CMA-ES. It directly addresses implicit ODEs and partial information by representing equations as D(u(t)) − F(t) = 0 and using a principled objective L_DE and L_SOL, augmented by a parsimony-driven criterion L_IC. Across three benchmarks that span explicit, implicit, and nonlinear dynamics, GODE generally achieves higher sample efficiency, better parsimony, and robust performance compared to state-of-the-art methods such as ODEFormer, PySR, and ProGED, demonstrating practical utility for modeling, system identification, and monitoring. The work suggests that grammar-based representations, combined with latent-space optimization, offer a scalable and interpretable pathway for dynamics discovery, with promising extensions to PDEs and multi-modal data in future research.

Abstract

The understanding and modeling of complex physical phenomena through dynamical systems has historically driven scientific progress, as it provides the tools for predicting the behavior of different systems under diverse conditions through time. The discovery of dynamical systems has been indispensable in engineering, as it allows for the analysis and prediction of complex behaviors for computational modeling, diagnostics, prognostics, and control of engineered systems. Joining recent efforts that harness the power of symbolic regression in this domain, we propose a novel framework for the end-to-end discovery of ordinary differential equations (ODEs), termed Grammar-based ODE Discovery Engine (GODE). The proposed methodology combines formal grammars with dimensionality reduction and stochastic search for efficiently navigating high-dimensional combinatorial spaces. Grammars allow us to seed domain knowledge and structure for both constraining, as well as, exploring the space of candidate expressions. GODE proves to be more sample- and parameter-efficient than state-of-the-art transformer-based models and to discover more accurate and parsimonious ODE expressions than both genetic programming- and other grammar-based methods for more complex inference tasks, such as the discovery of structural dynamics. Thus, we introduce a tool that could play a catalytic role in dynamics discovery tasks, including modeling, system identification, and monitoring tasks.

Figures (7)

• Figure 1: Overview of the end-to-end GODE process with formal grammars: First, grammars allow for an efficient generation of a dataset, then a VAE is trained to embed this dataset into a continuous latent space, which is searched during inference with a stochastic algorithm to identify the best-fitting ODE.
• Figure 2: Overview of the GVAE which encodes and decodes a sequence of rules.
• Figure 3: Violin plots of the relative L2 errors for the predicted time series based on the predicted equations with individual errors marked as orange dots for each investigated method.
• Figure 4: Relative L2 errors for the predicted time series based on the predicted equations of each example of the first benchmark.
• Figure 5: Solution trajectories of the predicted and ground truth ODEs of five selected examples from the second benchmark. The remaining examples can be found in \ref{['app:Bench2']}.