Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints

Alistair White, Niki Kilbertus, Maximilian Gelbrecht, Niklas Boers

TL;DR

The paper addresses learning dynamical systems from data under explicit constraints such as conservation laws or holonomic restrictions. It proposes Stabilized Neural Differential Equations (SNDEs), which augment the learned vector field with a stabilization term that enforces the constraint manifold $\mathcal{M} = \{u : g(u,t)=0\}$. Theoretical guarantees show that, with a stabilization matrix $F$ making $G(u)F(u)$ symmetric positive definite and a sufficiently large $\\gamma$, the manifold is asymptotically stable, while on $\mathcal{M}$ the stabilization vanishes and original dynamics are preserved. Empirically, SNDEs improve long-term fidelity and constraint satisfaction across autonomous and non-autonomous problems (two-body, rigid body, DC-DC converter, controlled robot arm, double pendulum), enabling reliable learning under conservation laws and controls with modest computational overhead.

Abstract

Many successful methods to learn dynamical systems from data have recently been introduced. However, ensuring that the inferred dynamics preserve known constraints, such as conservation laws or restrictions on the allowed system states, remains challenging. We propose stabilized neural differential equations (SNDEs), a method to enforce arbitrary manifold constraints for neural differential equations. Our approach is based on a stabilization term that, when added to the original dynamics, renders the constraint manifold provably asymptotically stable. Due to its simplicity, our method is compatible with all common neural differential equation (NDE) models and broadly applicable. In extensive empirical evaluations, we demonstrate that SNDEs outperform existing methods while broadening the types of constraints that can be incorporated into NDE training.

Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints

TL;DR

The paper addresses learning dynamical systems from data under explicit constraints such as conservation laws or holonomic restrictions. It proposes Stabilized Neural Differential Equations (SNDEs), which augment the learned vector field with a stabilization term that enforces the constraint manifold . Theoretical guarantees show that, with a stabilization matrix making symmetric positive definite and a sufficiently large , the manifold is asymptotically stable, while on the stabilization vanishes and original dynamics are preserved. Empirically, SNDEs improve long-term fidelity and constraint satisfaction across autonomous and non-autonomous problems (two-body, rigid body, DC-DC converter, controlled robot arm, double pendulum), enabling reliable learning under conservation laws and controls with modest computational overhead.

Abstract

Many successful methods to learn dynamical systems from data have recently been introduced. However, ensuring that the inferred dynamics preserve known constraints, such as conservation laws or restrictions on the allowed system states, remains challenging. We propose stabilized neural differential equations (SNDEs), a method to enforce arbitrary manifold constraints for neural differential equations. Our approach is based on a stabilization term that, when added to the original dynamics, renders the constraint manifold provably asymptotically stable. Due to its simplicity, our method is compatible with all common neural differential equation (NDE) models and broadly applicable. In extensive empirical evaluations, we demonstrate that SNDEs outperform existing methods while broadening the types of constraints that can be incorporated into NDE training.
Paper Structure (30 sections, 2 theorems, 32 equations, 8 figures, 4 tables)

This paper contains 30 sections, 2 theorems, 32 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. Hamiltonian and Lagrangian Neural Networks.
  5. Continuous Normalizing Flows on Riemannian Manifolds.
  6. Riemannian Optimization.
  7. Constraints via Regularization.
  8. Stabilized Neural Differential Equations
  9. General approach.
  10. Theoretical guarantees.
  11. Practical implementation.
  12. Results
  13. Two-Body Problem
  14. Motion of a Rigid Body
  15. DC-to-DC Converter
  16. ...and 15 more sections

Key Result

Theorem 1

Consider an NDE on an invariant manifold $\mathcal{M} = \{u \in {\mathbb{R}}^n \,;\, g(u) = 0\}$. A vector field $\dot u = h_{\theta}(u)$ admits all solutions of eq:prop1 on $\mathcal{M}$ if and only if $h_{\theta}|_{\mathcal{M}} = f_{\theta}|_{\mathcal{M}}.$

Figures (8)

  • Figure 1: Sketch of the basic idea behind stabilized neural differential equations (SNDEs). (a) An idealized, unstabilized NDE vector field (blue arrows). (b) A constraint manifold (black circle) and the corresponding stabilization of the vector field (pink arrows). (c) The overall, stabilized vector field of the SNDE (orange arrows) and a stabilized trajectory (green). The stabilization pushes any trajectory starting away from (but near) the manifold to converge to it at a rate $\gamma$ (see \ref{['sndes']}).
  • Figure 2: Top row: Results for the two-body problem experiment, showing a single test trajectory in (a) and averages over 100 test trajectories in (b-c). Middle row: Results for the rigid body rotation experiment, showing a single test trajectory in (d) and averages over 100 test trajectories in (e-f). Bottom row: Results for the DC-to-DC converter experiment, showing the voltage $v_1$ across the first capacitor during a single test trajectory in (g), and averages over 100 test trajectories in (h-i). The vanilla NODE (blue) is unstable in all settings, quickly drifting from the constraint manifold and subsequently diverging exponentially, while the vanilla ANODE (green) is unstable for the rigid body and DC-to-DC converter experiments. In contrast, the SNODE (red) and SANODE (purple) are constrained to the manifold with accurate predictions over a long horizon in all settings. Confidence intervals are not shown as they diverge along with the unstabilized trajectories.
  • Figure 3: Idealized schematic of a DC-to-DC converter.
  • Figure 4: Controlled robot arm. (a) Schematic of the robot arm. (b) Snapshot of a single test trajectory. After 100 seconds the NODE (blue) has drifted significantly from the prescribed control while the SNODE (red) accurately captures the ground truth dynamics (black). (c) Relative error in the endpoint $e(\theta)$ averaged over 100 test trajectories. The NODE (blue) accumulates errors and leaves the prescribed path, while the SNODE (red) remains accurate. Shadings in (c) are 95% confidence intervals.
  • Figure 5: Results for the double pendulum. (a) Relative error in the state over 300 short test trials, shown with 95% confidence intervals (shaded). Compared to the SNODE, the NODE diverges rapidly as it begins to accumulate errors in the energy. (b) Relative error in the energy averaged over 5 long test trials. (c) Comparison of the double pendulum's invariant measure estimated by the (hybrid) UDE, with and without stabilization, versus ground truth, with 95% confidence intervals.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Theorem 1: adapted from chin1995stabilization
  • Theorem 2: adapted from chin1995stabilization
  • proof