Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-informed neural networks via stochastic Hamiltonian dynamics learning

Chandrajit Bajaj, Minh Nguyen

TL;DR

The paper tackles optimal control for unknown dynamical systems by embedding the Pontryagin maximum principle (PMP) into neural learning. It develops discrete PMP-based planning and a continuous two-phase NeuralPMP framework that learns a reduced Hamiltonian $h(q,p)$ and employs forward-backward variational autoencoding to enhance exploration and robustness. The approach yields a practical method for inferring optimal trajectories via learned Hamiltonian dynamics, demonstrated on classical control tasks with uneven time steps and continuous-time settings. The results indicate competitive performance and improved exploration, with future directions including model-free extensions and scalability to higher-dimensional problems.

Abstract

In this paper, we propose novel learning frameworks to tackle optimal control problems by applying the Pontryagin maximum principle and then solving for a Hamiltonian dynamical system. Applying the Pontryagin maximum principle to the original optimal control problem shifts the learning focus to reduced Hamiltonian dynamics and corresponding adjoint variables. Then, the reduced Hamiltonian networks can be learned by going backwards in time and then minimizing loss function deduced from the Pontryagin maximum principle's conditions. The learning process is further improved by progressively learning a posterior distribution of the reduced Hamiltonians. This is achieved through utilizing a variational autoencoder which leads to more effective path exploration process. We apply our learning frameworks called NeuralPMP to various control tasks and obtain competitive results.

Physics-informed neural networks via stochastic Hamiltonian dynamics learning

TL;DR

The paper tackles optimal control for unknown dynamical systems by embedding the Pontryagin maximum principle (PMP) into neural learning. It develops discrete PMP-based planning and a continuous two-phase NeuralPMP framework that learns a reduced Hamiltonian and employs forward-backward variational autoencoding to enhance exploration and robustness. The approach yields a practical method for inferring optimal trajectories via learned Hamiltonian dynamics, demonstrated on classical control tasks with uneven time steps and continuous-time settings. The results indicate competitive performance and improved exploration, with future directions including model-free extensions and scalability to higher-dimensional problems.

Abstract

In this paper, we propose novel learning frameworks to tackle optimal control problems by applying the Pontryagin maximum principle and then solving for a Hamiltonian dynamical system. Applying the Pontryagin maximum principle to the original optimal control problem shifts the learning focus to reduced Hamiltonian dynamics and corresponding adjoint variables. Then, the reduced Hamiltonian networks can be learned by going backwards in time and then minimizing loss function deduced from the Pontryagin maximum principle's conditions. The learning process is further improved by progressively learning a posterior distribution of the reduced Hamiltonians. This is achieved through utilizing a variational autoencoder which leads to more effective path exploration process. We apply our learning frameworks called NeuralPMP to various control tasks and obtain competitive results.
Paper Structure (21 sections, 1 theorem, 13 equations, 4 figures, 4 tables)

This paper contains 21 sections, 1 theorem, 13 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Contributions
  4. Organization
  5. Optimal control formulation and PMP flow
  6. Optimal control formulation
  7. Dynamic programming principle
  8. Pontryagin maximum principle
  9. Discrete-time Hamiltonian dynamics learning
  10. Discrete framework with PMP-based loss
  11. Special linear quadratic control problem
  12. Continuous Hamiltonian dynamics learning with forward-backward variational autoencoder
  13. Reduced Hamiltonian learning with PMP-based training loss
  14. Second phase of variational training
  15. Experiments
  16. ...and 6 more sections

Key Result

theorem thmcountertheorem

(Pontryagin maximum principle) If $(q(.), u(.))$ is an optimal solution of the optimal control problem opt_formulation, then there exists the adjoint variable $p(.)$ of $q(.)$ so that:

Figures (4)

  • Figure 1: Discrete-time PMP training on LQR problem with uneven timesteps
  • Figure 2: Network architecture for phase 1 training
  • Figure 3: Network architecture for phase 2
  • Figure 4: Training benchmarks

Theorems & Definitions (1)

  • theorem thmcountertheorem