Learning System Dynamics without Forgetting

TL;DR

The paper tackles learning trajectories when system dynamics change across tasks by formalizing Continual Dynamics Learning (CDL). It introduces MS-GODE, a mode-switching graph ODE that uses per-task masks on a shared LG-ODE backbone to prevent forgetting, and Bio-CDL, a biologically diverse benchmark for evaluating dynamic-system learning. Empirical results on both physics simulations and Bio-CDL demonstrate that MS-GODE achieves higher average performance and substantially reduced forgetting compared to strong continual learning baselines, aided by its automatic mode-switching among masks. This work advances the ability to generalize dynamics learning across heterogeneous systems and provides a practical benchmark to drive future research in machine learning for dynamic systems.

Abstract

Observation-based trajectory prediction for systems with unknown dynamics is essential in fields such as physics and biology. Most existing approaches are limited to learning within a single system with fixed dynamics patterns. However, many real-world applications require learning across systems with evolving dynamics patterns, a challenge that has been largely overlooked. To address this, we systematically investigate the problem of Continual Dynamics Learning (CDL), examining task configurations and evaluating the applicability of existing techniques, while identifying key challenges. In response, we propose the Mode-switching Graph ODE (MS-GODE) model, which integrates the strengths LG-ODE and sub-network learning with a mode-switching module, enabling efficient learning over varying dynamics. Moreover, we construct a novel benchmark of biological dynamic systems for CDL, Bio-CDL, featuring diverse systems with disparate dynamics and significantly enriching the research field of machine learning for dynamic systems. Our code available at https://github.com/QueuQ/MS-GODE.

Figures (11)

• Figure 1: Illustration of the key components of one biological cellular system studied in our work: the RAN-regulated nucleocytoplasmic transport moore2013wrong. Briefly speaking, this model depicts the translocation of cargo proteins (Exportin) via nuclear pores with the assistance of RAN proteins. RAN is first activated (RAN*) and then binds to cargo forming a complex. Next, the complex is translocated across the nuclear membrane into the cytoplasm with the assistance of RAN. Finally, RAN and Exportin are dissociated.
• Figure 2: A molecule system may enter different phases and exhibit different dynamics as environmental factors (e.g. temperature) change. Molecules in solid state can only vibrate at fixed locations because of the strong interaction between them. Upon entering the liquid state, the interaction strength decreases and molecules can move around. In gas state, molecules move more freely with little molecule-wise interaction.
• Figure 3: The upper part illustrates the workflow of MS-GODE during mask selection and inference, which are denoted by dashed and solid lines. During mask selection, the observation is split into two parts, and the first part is fed into the model for selecting the mask that can best predict the second part. During inference, the entire observation is fed into the model for prediction of future states. The lower part demonstrates the structure of the masked encoder, masked generator, and masked decoder. Different components fetch their corresponding mask from the mask pool and apply the mask onto the parameters.
• Figure 4: Performance comparison among different strategies to binarize the mask values. (a) Comparison over the cellular system sequence $\mathrm{EGFR}_1 \rightarrow \mathrm{Ran}_1 \rightarrow \mathrm{EGFR}_2 \rightarrow \mathrm{Ran}_2$. (b)(c)(d) Comparison over different physics system sequences. Blue line denotes the performance of top-k selection with different thresholds. Red line demonstrates the performance of using fast selection.
• Figure 5: AP (a) and AF (b) of MS-GODE with different dropout rate.