Projected Neural Differential Equations for Learning Constrained Dynamics

Abstract

Neural differential equations offer a powerful approach for learning dynamics from data. However, they do not impose known constraints that should be obeyed by the learned model. It is well-known that enforcing constraints in surrogate models can enhance their generalizability and numerical stability. In this paper, we introduce projected neural differential equations (PNDEs), a new method for constraining neural differential equations based on projection of the learned vector field to the tangent space of the constraint manifold. In tests on several challenging examples, including chaotic dynamical systems and state-of-the-art power grid models, PNDEs outperform existing methods while requiring fewer hyperparameters. The proposed approach demonstrates significant potential for enhancing the modeling of constrained dynamical systems, particularly in complex domains where accuracy and reliability are essential.

Figures (6)

• Figure 1: Schematic of projected neural differential equations (PNDEs). The vector field of the unconstrained NDE (red arrow) is projected to the tangent space $T_u\mathcal{M}$ of the constraint manifold $\mathcal{M}$. For initial conditions $u_0 \in \mathcal{M}$, solutions (orange) of the projected vector field (green arrow) remain on the manifold and thereby satisfy the constraints.
• Figure 2: Performance of PNDEs compared to SNDEs and an unconstrained NDE for the Fermi–Pasta–Ulam–Tsingou (FPUT) lattice system fermi_studies_1955 with $N = 128$ segments. Panel (a) shows the ground truth as well as the NDE, SNDE, and PNDE predictions of the system state. Panel (b) shows the relative error. Panel (c) shows the error in the constrained quantity. Note that PNDE and SNDE results in (a) and (b) are visually indistinguishable.
• Figure 3: Illustration of a $3$-pendulum.
• Figure 4: Damped $N$-pendulum system. We compare the ability of an NDE to learn the dynamics of the damped $N$-pendulum system in generalized coordinates, where the constraints are satisfied automatically, versus Cartesian coordinates, where we explicitly enforce the constraints. Top: Across all systems, constrained models in Cartesian coordinates better predict the future evolution of the system's state. The difference in performance becomes more pronounced as $N$ is increased and the corresponding equations of motion become more complex in generalized coordinates. Bottom: PNDEs enforce the constraints exactly, while SNDEs incur large errors, especially for larger values of $N$. For SNDEs, smaller values of $\gamma$ than those shown here do not enforce the constraints, while larger values are computationally prohibitive due to stiffness. In generalized (angular) coordinates, the constraints are satisfied automatically by the choice of coordinate system. Nevertheless, the results show that converting to Cartesian coordinates and adding constraints leads to significantly better generalization. Unconstrained NDEs in Cartesian coordinates (not shown) are numerically unstable, highlighting the necessity of imposing explicit constraints via SNDEs or PNDEs.
• Figure 5: The IEEE 14-Bus system. Circles represent generators and arrows represent consumers. Diagram extracted from leon_quadratically_2020.

Theorems & Definitions (2)

• Proposition 1
• proof