Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems

Suresh Bishnoi, Ravinder Bhattoo, Sayan Ranu, N. M. Anoop Krishnan

TL;DR

This work tackles the challenge of learning physical dynamics that generalize to unseen system sizes by introducing GNODE, a graph-based neural ODE that embeds topology-aware inductive biases. Through a progressive sequence—CGnode, CDGnode, and finally MCGnode—the model explicitly enforces constraints and Newton's third law to conserve momentum and energy, yielding large gains in energy preservation and trajectory accuracy. Empirical results on $n$-pendulum and $n$-spring systems show MCGnode outperforms graph variants of Lagrangian and Hamiltonian networks by orders of magnitude and achieves strong zero-shot generalization, including 3D solids via peridynamics. The findings suggest that incorporating well-chosen inductive biases into NODE-based models can rival energy-conserving neural networks while maintaining computational efficiency and scalability.

Abstract

Neural networks with physics based inductive biases such as Lagrangian neural networks (LNN), and Hamiltonian neural networks (HNN) learn the dynamics of physical systems by encoding strong inductive biases. Alternatively, Neural ODEs with appropriate inductive biases have also been shown to give similar performances. However, these models, when applied to particle based systems, are transductive in nature and hence, do not generalize to large system sizes. In this paper, we present a graph based neural ODE, GNODE, to learn the time evolution of dynamical systems. Further, we carefully analyse the role of different inductive biases on the performance of GNODE. We show that, similar to LNN and HNN, encoding the constraints explicitly can significantly improve the training efficiency and performance of GNODE significantly. Our experiments also assess the value of additional inductive biases, such as Newtons third law, on the final performance of the model. We demonstrate that inducing these biases can enhance the performance of model by orders of magnitude in terms of both energy violation and rollout error. Interestingly, we observe that the GNODE trained with the most effective inductive biases, namely MCGNODE, outperforms the graph versions of LNN and HNN, namely, Lagrangian graph networks (LGN) and Hamiltonian graph networks (HGN) in terms of energy violation error by approx 4 orders of magnitude for a pendulum system, and approx 2 orders of magnitude for spring systems. These results suggest that competitive performances with energy conserving neural networks can be obtained for NODE based systems by inducing appropriate inductive biases.

Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems

TL;DR

This work tackles the challenge of learning physical dynamics that generalize to unseen system sizes by introducing GNODE, a graph-based neural ODE that embeds topology-aware inductive biases. Through a progressive sequence—CGnode, CDGnode, and finally MCGnode—the model explicitly enforces constraints and Newton's third law to conserve momentum and energy, yielding large gains in energy preservation and trajectory accuracy. Empirical results on -pendulum and -spring systems show MCGnode outperforms graph variants of Lagrangian and Hamiltonian networks by orders of magnitude and achieves strong zero-shot generalization, including 3D solids via peridynamics. The findings suggest that incorporating well-chosen inductive biases into NODE-based models can rival energy-conserving neural networks while maintaining computational efficiency and scalability.

Abstract

Neural networks with physics based inductive biases such as Lagrangian neural networks (LNN), and Hamiltonian neural networks (HNN) learn the dynamics of physical systems by encoding strong inductive biases. Alternatively, Neural ODEs with appropriate inductive biases have also been shown to give similar performances. However, these models, when applied to particle based systems, are transductive in nature and hence, do not generalize to large system sizes. In this paper, we present a graph based neural ODE, GNODE, to learn the time evolution of dynamical systems. Further, we carefully analyse the role of different inductive biases on the performance of GNODE. We show that, similar to LNN and HNN, encoding the constraints explicitly can significantly improve the training efficiency and performance of GNODE significantly. Our experiments also assess the value of additional inductive biases, such as Newtons third law, on the final performance of the model. We demonstrate that inducing these biases can enhance the performance of model by orders of magnitude in terms of both energy violation and rollout error. Interestingly, we observe that the GNODE trained with the most effective inductive biases, namely MCGNODE, outperforms the graph versions of LNN and HNN, namely, Lagrangian graph networks (LGN) and Hamiltonian graph networks (HGN) in terms of energy violation error by approx 4 orders of magnitude for a pendulum system, and approx 2 orders of magnitude for spring systems. These results suggest that competitive performances with energy conserving neural networks can be obtained for NODE based systems by inducing appropriate inductive biases.
Paper Structure (34 sections, 1 theorem, 20 equations, 23 figures, 4 tables)

This paper contains 34 sections, 1 theorem, 20 equations, 23 figures, 4 tables.

Table of Contents

  1. Introduction and related works
  2. Background on dynamical systems
  3. Inductive biases for neural ODE
  4. Graph neural ODE (Gnode) for physical systems
  5. Decoupling the dynamics and constraints (CGnode).
  6. Decoupling the internal and body forces (CDGnode)
  7. Newton's third law (MCGnode)
  8. Empirical Evaluation
  9. Experimental setup
  10. Evaluation of inductive biases in Gnode
  11. Comparison with baselines
  12. Generalizability
  13. Learning the dynamics of 3D solid
  14. Robustness to Noise
  15. Conclusions
  16. ...and 19 more sections

Key Result

Theorem 1

In the absence of an external field, MCGnode exactly conserves the momentum.

Figures (23)

  • Figure 1: Architecture of CDGnode (see App. \ref{['app:diff_graph_arch']}).
  • Figure 2: Comparison of the energy violation and rollout error between Node, Gnode, CGnode, CDGnode, MCGnode for $n$-pendulum and spring systems with $n=3,4,5$ links. For the pendulum system, the model is trained on the 5-pendulum system and for the spring system, the model is trained on the 5-spring system and evaluated on all other systems. The shaded region shows the 95% confidence interval from 100 initial conditions.
  • Figure 3: Comparison of the energy violation and rollout error between Gnode, Lgn, and Hgn for $n$-pendulum and spring systems with $n=3,4,5$ links. For both the pendulum and spring systems, the model is trained on a 5-link structure and evaluated on all other systems. The shaded region shows the 95% confidence interval from 100 initial conditions.
  • Figure 4: Momentum error ($ME$) of Node, Gnode, CGnode, CDGnode, and MCGnode (top palette), and Lgn, Hgn, MCGnode (bottom palette) for spring (top row of both the palettes) and pendulum (bottom row of both the palettes) systems with 3, 4, and 5 links. The shaded region represents the 95% confidence interval based on 100 forward simulations.
  • Figure 5: Energy violation and rollout error between Gnode, Lgn, and Hgn for spring systems with $5, 20, 50$ links and pendulum system with $5, 10, 20$ links. Note that for all the systems, Gnode, Lgn, and Hgn are trained on 5-pendulum and 5-spring systems and tested on others. The shaded region represents the $95\%$ confidence interval based on $100$ forward simulations.
  • ...and 18 more figures

Theorems & Definitions (1)

  • Theorem 1