Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-Informed Regularization for Domain-Agnostic Dynamical System Modeling

Zijie Huang, Wanjia Zhao, Jingdong Gao, Ziniu Hu, Xiao Luo, Yadi Cao, Yuanzhou Chen, Yizhou Sun, Wei Wang

TL;DR

While TRS is a domain-specific physical prior, this work presents the first theoretical proof that TRS loss can universally improve modeling accuracy by minimizing higher-order Taylor terms in ODE integration, which is numerically beneficial to various systems regardless of their properties, even for irreversible systems.

Abstract

Learning complex physical dynamics purely from data is challenging due to the intrinsic properties of systems to be satisfied. Incorporating physics-informed priors, such as in Hamiltonian Neural Networks (HNNs), achieves high-precision modeling for energy-conservative systems. However, real-world systems often deviate from strict energy conservation and follow different physical priors. To address this, we present a framework that achieves high-precision modeling for a wide range of dynamical systems from the numerical aspect, by enforcing Time-Reversal Symmetry (TRS) via a novel regularization term. It helps preserve energies for conservative systems while serving as a strong inductive bias for non-conservative, reversible systems. While TRS is a domain-specific physical prior, we present the first theoretical proof that TRS loss can universally improve modeling accuracy by minimizing higher-order Taylor terms in ODE integration, which is numerically beneficial to various systems regardless of their properties, even for irreversible systems. By integrating the TRS loss within neural ordinary differential equation models, the proposed model TREAT demonstrates superior performance on diverse physical systems. It achieves a significant 11.5% MSE improvement in a challenging chaotic triple-pendulum scenario, underscoring TREAT's broad applicability and effectiveness.

Physics-Informed Regularization for Domain-Agnostic Dynamical System Modeling

TL;DR

While TRS is a domain-specific physical prior, this work presents the first theoretical proof that TRS loss can universally improve modeling accuracy by minimizing higher-order Taylor terms in ODE integration, which is numerically beneficial to various systems regardless of their properties, even for irreversible systems.

Abstract

Learning complex physical dynamics purely from data is challenging due to the intrinsic properties of systems to be satisfied. Incorporating physics-informed priors, such as in Hamiltonian Neural Networks (HNNs), achieves high-precision modeling for energy-conservative systems. However, real-world systems often deviate from strict energy conservation and follow different physical priors. To address this, we present a framework that achieves high-precision modeling for a wide range of dynamical systems from the numerical aspect, by enforcing Time-Reversal Symmetry (TRS) via a novel regularization term. It helps preserve energies for conservative systems while serving as a strong inductive bias for non-conservative, reversible systems. While TRS is a domain-specific physical prior, we present the first theoretical proof that TRS loss can universally improve modeling accuracy by minimizing higher-order Taylor terms in ODE integration, which is numerically beneficial to various systems regardless of their properties, even for irreversible systems. By integrating the TRS loss within neural ordinary differential equation models, the proposed model TREAT demonstrates superior performance on diverse physical systems. It achieves a significant 11.5% MSE improvement in a challenging chaotic triple-pendulum scenario, underscoring TREAT's broad applicability and effectiveness.
Paper Structure (43 sections, 3 theorems, 60 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 43 sections, 3 theorems, 60 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. NeuralODE for Dynamical Systems
  4. Time-Reversal Symmetry (TRS)
  5. Method: TREAT
  6. Time-Reversal Symmetry Loss and Training
  7. Forward Trajectory Prediction and Reconstruction Loss.
  8. Reverse Trajectory Prediction and Regularization Loss.
  9. Theoretical Analysis of Time-Reversal Symmetry Loss
  10. Experiments
  11. Main Results
  12. Ablation and Sensitivity Analysis
  13. Visualizations
  14. Conclusions
  15. Limitations
  16. ...and 28 more sections

Key Result

Lemma 2.1

Eqn eq:def_reverse is equivalent to $\ R \circ \phi_t \circ R \circ \phi_t=I$, where $I$ denotes identity mapping.

Figures (9)

  • Figure 1: (a) High-precision modeling for dynamical systems; (b.1) Classification of classical mechanical systems based on Tolman1938TheDOlamb1998time;(b.2) Tim-Reversal Symmetry illustration;(b.3) Error accumulation in numerical solvers.
  • Figure 2: Illustration of time-reversal symmetry based on Lemma \ref{['lemma:timereversal']}.The total length of the trajectory is $t_K-t_0 = T$. $t_k'$ is the time index in the reverse trajectory, which points to the same time as $t_{K-k}$ in the forward trajectory.
  • Figure 3: Overall framework of TREAT. $O_1, O_2, O_3$ are connected agents. It follows the encoder-processor-decoder architecture introduced in Sec \ref{['sec:prelim_graphode']}. A novel TRS loss is incorporated to improve modeling accuracy across systems from the numerical aspect, regardless of their physical properties.
  • Figure 4: Varying prediction lengths across multi-agent datasets (Pendulum MSE is in log values).
  • Figure 5: Varying $\alpha$ values across multi-agent datasets.
  • ...and 4 more figures

Theorems & Definitions (4)

  • Lemma 2.1
  • Theorem 3.1
  • Lemma 3.2
  • proof