Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PGODE: Towards High-quality System Dynamics Modeling

Xiao Luo, Yiyang Gu, Huiyu Jiang, Hang Zhou, Jinsheng Huang, Wei Ju, Zhiping Xiao, Ming Zhang, Yizhou Sun

TL;DR

The core of PGODE is to incorporate prototype decomposition from contextual knowledge into a continuous graph ODE framework to extract both object-level and system-level contexts from historical trajectories and enhance the generalization capability under system changes.

Abstract

This paper studies the problem of modeling multi-agent dynamical systems, where agents could interact mutually to influence their behaviors. Recent research predominantly uses geometric graphs to depict these mutual interactions, which are then captured by powerful graph neural networks (GNNs). However, predicting interacting dynamics in challenging scenarios such as out-of-distribution shift and complicated underlying rules remains unsolved. In this paper, we propose a new approach named Prototypical Graph ODE (PGODE) to address the problem. The core of PGODE is to incorporate prototype decomposition from contextual knowledge into a continuous graph ODE framework. Specifically, PGODE employs representation disentanglement and system parameters to extract both object-level and system-level contexts from historical trajectories, which allows us to explicitly model their independent influence and thus enhances the generalization capability under system changes. Then, we integrate these disentangled latent representations into a graph ODE model, which determines a combination of various interacting prototypes for enhanced model expressivity. The entire model is optimized using an end-to-end variational inference framework to maximize the likelihood. Extensive experiments in both in-distribution and out-of-distribution settings validate the superiority of PGODE compared to various baselines.

PGODE: Towards High-quality System Dynamics Modeling

TL;DR

The core of PGODE is to incorporate prototype decomposition from contextual knowledge into a continuous graph ODE framework to extract both object-level and system-level contexts from historical trajectories and enhance the generalization capability under system changes.

Abstract

This paper studies the problem of modeling multi-agent dynamical systems, where agents could interact mutually to influence their behaviors. Recent research predominantly uses geometric graphs to depict these mutual interactions, which are then captured by powerful graph neural networks (GNNs). However, predicting interacting dynamics in challenging scenarios such as out-of-distribution shift and complicated underlying rules remains unsolved. In this paper, we propose a new approach named Prototypical Graph ODE (PGODE) to address the problem. The core of PGODE is to incorporate prototype decomposition from contextual knowledge into a continuous graph ODE framework. Specifically, PGODE employs representation disentanglement and system parameters to extract both object-level and system-level contexts from historical trajectories, which allows us to explicitly model their independent influence and thus enhances the generalization capability under system changes. Then, we integrate these disentangled latent representations into a graph ODE model, which determines a combination of various interacting prototypes for enhanced model expressivity. The entire model is optimized using an end-to-end variational inference framework to maximize the likelihood. Extensive experiments in both in-distribution and out-of-distribution settings validate the superiority of PGODE compared to various baselines.
Paper Structure (31 sections, 4 theorems, 40 equations, 6 figures, 15 tables, 1 algorithm)

This paper contains 31 sections, 4 theorems, 40 equations, 6 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. The Proposed Approach
  4. Hierarchical Context Discovery with Disentanglement
  5. Prototypical Graph ODE
  6. Decoder and Optimization
  7. Experiment
  8. Performance on Physical Dynamics Simulations
  9. Performance on Molecular Dynamics Simulations
  10. Further Analysis
  11. Related Work
  12. Interacting Dynamics Modeling
  13. Neural Ordinary Differential Equations
  14. Conclusion
  15. Algorithm
  16. ...and 16 more sections

Key Result

Theorem 3.1

Assume the prototype function $\psi_a^k$ has a bounded gradient. Moreover, each prototype function $\psi_a^k$ and $\psi_r^k$ are Lipschitz continuous with Lipschitz constant $L^k_{a}$ and $L^k_r$, and $\psi_a$ and $\psi_r$ are for single prototype function with Lipschitz constant $L_a$ and $L_r$. Fo

Figures (6)

  • Figure 1: An overview of the proposed PGODE. Our PGODE first constructs a temporal graph and then utilizes different encoders to extract object-level and system-level contexts using representation disentanglement and system parameters. These contexts would generate weights for a prototypical graph ODE framework, which models the evolution of interacting objects. In the end, the latent states of objects are fed into a decoder to output the trajectories at any timestamp.
  • Figure 2: Visualization of different methods on Springs. Semi-transparent paths denote observed trajectories and solid paths represent our predictions.
  • Figure 3: Visualization of prediction results of different methods on the 5AWL dataset. We can observe that our PGODE can reconstruct the ground truth accurately.
  • Figure 4: (a), (b) Performance with respect to varying condition lengths on Springs and 5AWL. (c) (d) Performance and running time with respect to different numbers of prototypes.
  • Figure 5: (a),(b),(c),(d) Performance on the OOD test set of Springs, Charged, 5AWL, and 2N5C with respect to four different numbers of prototypes. (e),(f) Performance with respect to different condition lengths on the ID test set of Springs and 5AWL.
  • ...and 1 more figures

Theorems & Definitions (7)

  • Theorem 3.1
  • Lemma 3.2
  • proof
  • Theorem 3.1
  • Lemma 3.2
  • proof
  • proof