Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Control: Concurrent System Identification and Control Learning with Neural ODE

Cheng Chi

TL;DR

The paper addresses controlling unknown continuous-time dynamical systems by jointly learning dynamics and optimal control via a coupled Neural ODE (NC). The approach replaces separate system identification with end-to-end training of two interacting networks: a dynamics learner $g_{\gamma}$ and a controller $h_{\theta}$, linked through $\dot{x}=g_{\gamma}(x, h_{\theta}(x,t), t)$. Training optimizes both trajectory fidelity to real dynamics and adherence to the control objective, using an alternating training scheme to progressively improve both components. Demonstrations on a linear Ax+Bu system and the CartPole task show data-efficient learning, achieving near-optimal control with significantly fewer real trajectories than typical reinforcement learning methods. This integration promises practical impact for unknown-dynamics control where data is scarce or expensive to collect.

Abstract

Controlling continuous-time dynamical systems is generally a two step process: first, identify or model the system dynamics with differential equations, then, minimize the control objectives to achieve optimal control function and optimal state trajectories. However, any inaccuracy in dynamics modeling will lead to sub-optimality in the resulting control function. To address this, we propose a neural ODE based method for controlling unknown dynamical systems, denoted as Neural Control (NC), which combines dynamics identification and optimal control learning using a coupled neural ODE. Through an intriguing interplay between the two neural networks in coupled neural ODE structure, our model concurrently learns system dynamics as well as optimal controls that guides towards target states. Our experiments demonstrate the effectiveness of our model for learning optimal control of unknown dynamical systems. Codes available at https://github.com/chichengmessi/neural_ode_control/tree/main

Neural Control: Concurrent System Identification and Control Learning with Neural ODE

TL;DR

The paper addresses controlling unknown continuous-time dynamical systems by jointly learning dynamics and optimal control via a coupled Neural ODE (NC). The approach replaces separate system identification with end-to-end training of two interacting networks: a dynamics learner and a controller , linked through . Training optimizes both trajectory fidelity to real dynamics and adherence to the control objective, using an alternating training scheme to progressively improve both components. Demonstrations on a linear Ax+Bu system and the CartPole task show data-efficient learning, achieving near-optimal control with significantly fewer real trajectories than typical reinforcement learning methods. This integration promises practical impact for unknown-dynamics control where data is scarce or expensive to collect.

Abstract

Controlling continuous-time dynamical systems is generally a two step process: first, identify or model the system dynamics with differential equations, then, minimize the control objectives to achieve optimal control function and optimal state trajectories. However, any inaccuracy in dynamics modeling will lead to sub-optimality in the resulting control function. To address this, we propose a neural ODE based method for controlling unknown dynamical systems, denoted as Neural Control (NC), which combines dynamics identification and optimal control learning using a coupled neural ODE. Through an intriguing interplay between the two neural networks in coupled neural ODE structure, our model concurrently learns system dynamics as well as optimal controls that guides towards target states. Our experiments demonstrate the effectiveness of our model for learning optimal control of unknown dynamical systems. Codes available at https://github.com/chichengmessi/neural_ode_control/tree/main
Paper Structure (15 sections, 9 equations, 9 figures, 1 algorithm)

This paper contains 15 sections, 9 equations, 9 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Classical optimal control
  4. Neural ODE
  5. Neural Control: NC
  6. Model architecture
  7. Training Loss
  8. Alternative Training
  9. Experiments
  10. Control task: Linear System
  11. Control Task: CartPole
  12. Conclusion
  13. Alternative training motivation
  14. Compare with analytical optimal control
  15. Dynamic learner and Controller training loss

Figures (9)

  • Figure 1: Training loss for dynamics learner $g_{\gamma}$ and controller $h_{\theta}$
  • Figure 2: The $k_{th}$ alternation of the dynamics learner ${g}_{\gamma}$ and controller ${h}_{\theta}$ alternative training
  • Figure 3: The full training process of NC for Ax+Bu task
  • Figure 4: CartPole dynamical system
  • Figure 5: The full training process of NC for CartPole task
  • ...and 4 more figures