Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Path Integral Control for Hybrid Dynamical Systems

Hongzhe Yu, Diana Frias Franco, Aaron M. Johnson, Yongxin Chen

TL;DR

This work introduces a novel paradigm for solving optimal control problems for hybrid dynamical systems under uncertainties by proposing the Hybrid Path Integral (H-PI) framework and showing that the optimal controller can be obtained by evaluating a path integral with hybrid constraints.

Abstract

This work introduces a novel paradigm for solving optimal control problems for hybrid dynamical systems under uncertainties. Robotic systems having contact with the environment can be modeled as hybrid systems. Controller design for hybrid systems under disturbances is complicated by the discontinuous jump dynamics, mode changes with inconsistent state dimensions, and variations in jumping timing and states caused by noise. We formulate this problem into a stochastic control problem with hybrid transition constraints and propose the Hybrid Path Integral (H-PI) framework to obtain the optimal controller. Despite random mode changes across stochastic path samples, we show that the ratio between hybrid path distributions with varying drift terms remains analogous to the smooth path distributions. We then show that the optimal controller can be obtained by evaluating a path integral with hybrid constraints. Importance sampling for path distributions with hybrid dynamics constraints is introduced to reduce the variance of the path integral evaluation, where we leverage the recently developed Hybrid iterative-Linear-Quadratic-Regulator (H-iLQR) controller to induce a hybrid path distribution proposal with low variance. The proposed method is validated through numerical experiments on various hybrid systems and extensive ablation studies. All the sampling processes are conducted in parallel on a Graphics Processing Unit (GPU).

Path Integral Control for Hybrid Dynamical Systems

TL;DR

This work introduces a novel paradigm for solving optimal control problems for hybrid dynamical systems under uncertainties by proposing the Hybrid Path Integral (H-PI) framework and showing that the optimal controller can be obtained by evaluating a path integral with hybrid constraints.

Abstract

This work introduces a novel paradigm for solving optimal control problems for hybrid dynamical systems under uncertainties. Robotic systems having contact with the environment can be modeled as hybrid systems. Controller design for hybrid systems under disturbances is complicated by the discontinuous jump dynamics, mode changes with inconsistent state dimensions, and variations in jumping timing and states caused by noise. We formulate this problem into a stochastic control problem with hybrid transition constraints and propose the Hybrid Path Integral (H-PI) framework to obtain the optimal controller. Despite random mode changes across stochastic path samples, we show that the ratio between hybrid path distributions with varying drift terms remains analogous to the smooth path distributions. We then show that the optimal controller can be obtained by evaluating a path integral with hybrid constraints. Importance sampling for path distributions with hybrid dynamics constraints is introduced to reduce the variance of the path integral evaluation, where we leverage the recently developed Hybrid iterative-Linear-Quadratic-Regulator (H-iLQR) controller to induce a hybrid path distribution proposal with low variance. The proposed method is validated through numerical experiments on various hybrid systems and extensive ablation studies. All the sampling processes are conducted in parallel on a Graphics Processing Unit (GPU).

Paper Structure

This paper contains 30 sections, 3 theorems, 74 equations, 11 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Contributions
  4. Preliminaries
  5. Smooth Path Integral Control
  6. Hybrid Systems and Jump Dynamics Linearization
  7. Problem Formulation
  8. Stochastic Systems with Hybrid Transitions
  9. Stochastic Optimal Control with Hybrid Transitions
  10. Path Integral Control for Hybrid Systems
  11. Ratio Between Hybrid Path Distribution Measures
  12. Hybrid Path Distribution Control
  13. Optimal Controller as Hybrid Path Integral
  14. Hybrid Path Distribution Importance Sampling
  15. Importance Sampling in Hybrid Path Distribution Space
  16. ...and 15 more sections

Key Result

Lemma 1

The ratio between the measures $d \mathbb{Q}_H$ and $d \mathbb{P}_H$ is given by where $\{(t_j^{-}, t_j^{+})\}_{N_J}, t_{N_J+1}^{-} = T$ are the jump times of a rollout of hybrid system eq:nonlinear_SDE_1, eq:reset_map_j under measure $d\mathbb{P}_H$.

Figures (11)

  • Figure 1: Hybrid Path Integral Sampling Procedure for Hybrid Dynamical Systems with Stochastic Smooth Flows.
  • Figure 2: Samples from the controlled (left) and uncontrolled (right) system state distributions for the bouncing ball dynamics. Lemma \ref{['thm:change_of_measure']} states that the KL-divergence between the two distributions equals the expected control energy.
  • Figure 3: A visualization of the proposed H-PI-iLQR method for the bouncing ball dynamics example.
  • Figure 4: Controlled trajectories under H-PI and H-iLQR for the best and tail $5 \%$ costs attained by H-iLQR. Near the jumping events, larger deviations appear under H-iLQR, inducing larger control energy costs.
  • Figure 5: Controlled trajectories of the SLIP jumping experiment, for the tail $5\%$ cost attained by H-iLQR.
  • ...and 6 more figures

Theorems & Definitions (7)

  • Lemma 1
  • proof
  • Corollary 1
  • Lemma 2
  • proof
  • proof
  • proof