Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting symbolic ODEs from multiple trajectories

Yakup Emre Şahin, Niki Kilbertus, Sören Becker

TL;DR

Predicting symbolic ODEs from multiple trajectories, the paper introduces MIO, a transformer-based model that uses multiple instance learning to aggregate information across several observed trajectories and infer a single symbolic ODE. By extending ODEFormer with a learnable aggregator between the encoder and decoder, MIO achieves robust identification across 2D–4D systems and varying noise levels, with simple mean pooling performing competitively against more complex strategies. In experiments against equation-based baselines (PySR, SINDy, FFX) and a single-instance transformer baseline (ODEFormer), MIO consistently outperforms and gains most when moving from one to two instances, highlighting the value of multi-trajectory information. The findings emphasize that simple aggregation can suffice for multi-instance ODE discovery and underscore practical benefits for scientific modeling where repeated measures are available; limitations include distributional assumptions and scalability, guiding future work toward more generalizable and higher-dimensional setups.

Abstract

We introduce MIO, a transformer-based model for inferring symbolic ordinary differential equations (ODEs) from multiple observed trajectories of a dynamical system. By combining multiple instance learning with transformer-based symbolic regression, the model effectively leverages repeated observations of the same system to learn more generalizable representations of the underlying dynamics. We investigate different instance aggregation strategies and show that even simple mean aggregation can substantially boost performance. MIO is evaluated on systems ranging from one to four dimensions and under varying noise levels, consistently outperforming existing baselines.

Predicting symbolic ODEs from multiple trajectories

TL;DR

Predicting symbolic ODEs from multiple trajectories, the paper introduces MIO, a transformer-based model that uses multiple instance learning to aggregate information across several observed trajectories and infer a single symbolic ODE. By extending ODEFormer with a learnable aggregator between the encoder and decoder, MIO achieves robust identification across 2D–4D systems and varying noise levels, with simple mean pooling performing competitively against more complex strategies. In experiments against equation-based baselines (PySR, SINDy, FFX) and a single-instance transformer baseline (ODEFormer), MIO consistently outperforms and gains most when moving from one to two instances, highlighting the value of multi-trajectory information. The findings emphasize that simple aggregation can suffice for multi-instance ODE discovery and underscore practical benefits for scientific modeling where repeated measures are available; limitations include distributional assumptions and scalability, guiding future work toward more generalizable and higher-dimensional setups.

Abstract

We introduce MIO, a transformer-based model for inferring symbolic ordinary differential equations (ODEs) from multiple observed trajectories of a dynamical system. By combining multiple instance learning with transformer-based symbolic regression, the model effectively leverages repeated observations of the same system to learn more generalizable representations of the underlying dynamics. We investigate different instance aggregation strategies and show that even simple mean aggregation can substantially boost performance. MIO is evaluated on systems ranging from one to four dimensions and under varying noise levels, consistently outperforming existing baselines.
Paper Structure (12 sections, 1 equation, 5 figures, 1 table)

This paper contains 12 sections, 1 equation, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Results
  4. Conclusion
  5. Acknowledgements
  6. Details of data generation
  7. Details on MIO: architecture, training, inference
  8. Model architecture
  9. Experiment 1
  10. Experiment 2
  11. Details on baseline models
  12. Additional results

Figures (5)

  • Figure 1: Model overview. System dimensions (2D, 3D, 4D) and # instances (2N, 2N, 1N) may vary.
  • Figure 2: Performance comparison of different instance aggregation methods.
  • Figure 3: Performance comparison across system dimensions, number of instances and noise levels.
  • Figure 4: Heatmap of different aggregation methods. We demonstrate how the performance of each aggregation approach depends on number of dimensions and number of instances.
  • Figure 5: Multitraj-ODEFormer and baselines in 2D and 3D scheme. We compare our models trained with different number of instances with baseline models in reconstruction task.