Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Optimal Control Problems of Rigid-Body Dynamics with Collisions Using the Hybrid Minimum Principle

Wei Hu, Jihao Long, Yaohua Zang, Weinan E, Jiequn Han

TL;DR

This work tackles open-loop optimal control for rigid-body systems with autonomous state jumps caused by collisions, modeling these jumps through a collision detector $\psi(m{x})$ and a jump map $m{g}$. It extends Pontryagin’s maximum principle to the hybrid setting (HMP) and couples it with a relaxed method of successive approximations (MSA) to compute controls, using forward-Euler integration and careful handling of discontinuities such as collision-time estimation and jump propagation. The authors demonstrate linear convergence with respect to iteration steps and first-order accuracy in time discretization on disc-collision problems, and show that their HMP-based method outperforms direct gradient methods and deep reinforcement learning in accuracy and efficiency. The approach is robust to multiple collisions, supports nonquadratic running costs, and provides analytic benchmarks for benchmarking and future extensions in more complex hybrid robotic systems.

Abstract

Collisions are common in many dynamical systems with real applications. They can be formulated as hybrid dynamical systems with discontinuities automatically triggered when states transverse certain manifolds. We present an algorithm for the optimal control problem of such hybrid dynamical systems based on solving the equations derived from the hybrid minimum principle (HMP). The algorithm is an iterative scheme following the spirit of the method of successive approximations (MSA), and it is robust to undesired collisions observed in the initial guesses. We propose several techniques to address the additional numerical challenges introduced by the presence of discontinuities. The algorithm is tested on disc collision problems whose optimal solutions exhibit one or multiple collisions. Linear convergence in terms of iteration steps and asymptotic first-order accuracy in terms of time discretization are observed when the algorithm is implemented with the forward-Euler scheme. The numerical results demonstrate that the proposed algorithm has better accuracy and convergence than direct methods based on gradient descent. Furthermore, the algorithm is also simpler, more accurate, and more stable than a deep reinforcement learning method.

Solving Optimal Control Problems of Rigid-Body Dynamics with Collisions Using the Hybrid Minimum Principle

TL;DR

This work tackles open-loop optimal control for rigid-body systems with autonomous state jumps caused by collisions, modeling these jumps through a collision detector and a jump map . It extends Pontryagin’s maximum principle to the hybrid setting (HMP) and couples it with a relaxed method of successive approximations (MSA) to compute controls, using forward-Euler integration and careful handling of discontinuities such as collision-time estimation and jump propagation. The authors demonstrate linear convergence with respect to iteration steps and first-order accuracy in time discretization on disc-collision problems, and show that their HMP-based method outperforms direct gradient methods and deep reinforcement learning in accuracy and efficiency. The approach is robust to multiple collisions, supports nonquadratic running costs, and provides analytic benchmarks for benchmarking and future extensions in more complex hybrid robotic systems.

Abstract

Collisions are common in many dynamical systems with real applications. They can be formulated as hybrid dynamical systems with discontinuities automatically triggered when states transverse certain manifolds. We present an algorithm for the optimal control problem of such hybrid dynamical systems based on solving the equations derived from the hybrid minimum principle (HMP). The algorithm is an iterative scheme following the spirit of the method of successive approximations (MSA), and it is robust to undesired collisions observed in the initial guesses. We propose several techniques to address the additional numerical challenges introduced by the presence of discontinuities. The algorithm is tested on disc collision problems whose optimal solutions exhibit one or multiple collisions. Linear convergence in terms of iteration steps and asymptotic first-order accuracy in terms of time discretization are observed when the algorithm is implemented with the forward-Euler scheme. The numerical results demonstrate that the proposed algorithm has better accuracy and convergence than direct methods based on gradient descent. Furthermore, the algorithm is also simpler, more accurate, and more stable than a deep reinforcement learning method.
Paper Structure (23 sections, 1 theorem, 61 equations, 15 figures, 1 table, 1 algorithm)

This paper contains 23 sections, 1 theorem, 61 equations, 15 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed method and contributions
  3. Organization and notations
  4. Problem formulation
  5. Hybrid minimum principle
  6. Algorithm
  7. The relaxed MSA algorithm
  8. Experiments
  9. Concentric collision
  10. General situation for two discs
  11. Multiple collisions
  12. L1 running cost
  13. Convergence rate with $\alpha$
  14. Comparisons to other methods
  15. Comparison to HMP-MAS
  16. ...and 8 more sections

Key Result

Theorem 3.1

Let the state-control pair $\{\bm{x}(t), \bm{u}(t)\}$ be the optimal solution associated with eq:opt_problem, then there exists a costate process $\bm{\lambda}(t): [0,T]\rightarrow\mathbb{R}^m$ that satisfies for $i=0, \cdots, M$, with the terminal condition and backward jump conditions, $i=1, \cdots, M,$ where $\eta_i \in \mathbb{R}$ satisfies the following equation (variation of collision time

Figures (15)

  • Figure 5.1: Illustration of the concentric collision in \ref{['sub:centric_collision']}. \ref{['fig:case3_illustration']} Initial state: The initial locations of discs are the blue discs. The red line and green line are optimal trajectories of disc 1 and disc 2, respectively. The dashed red circle is the before-collision position of disc 1. The orange point is the collision point. \ref{['fig:case3_shape']} The numerical optimal control and the analytical optimal control, $\bm{u}=[u_x, u_y]$.
  • Figure 5.2: Convergence of the iterative algorithm on the concentric collision example in \ref{['sub:centric_collision']}. The annotations on the right are the final values of the corresponding curves. The common $x$ axis denotes the number of iterations spent in the \ref{['alg:PMP']}. Top: the difference between the analytical optimal objective and the numerical solutions. Middle: the $L^2$ semi-norm of the derivative of the Hamiltonian with respect to control $\bm{u}$. The dashed line is the result of the regression in the form $\mu\sigma^k$ (this is the linear regression after taking the logarithm. $k$ is the number of iterations, $\mu,\sigma$ are positive regression parameters). The inset plot is the ratio $\vert H_{u}^{k+1} \vert / \vert H_{u}^{k} \vert$ versus the number of iterations. It suggests that the rate of linear convergence is approximately $0.99$. Bottom: the difference between the analytical optimal control and the numerical solutions in terms of the $L^2$ semi-norm.
  • Figure 5.3: Convergence to the analytical solution when the number of time intervals $N$ tends to infinity on the concentric collision example in \ref{['sub:centric_collision']}. The dashed orange lines are straight lines obtained from the least square regression on the data with model $\sigma/N$ and parameter $\sigma$. Left: Difference between the simulated collision times and the analytical optimal collision time. Middle: Difference between the numerical and analytical solutions for the optimal controls in terms of the $L^2$ semi-norm. Right: Difference between the simulated total costs and the analytical solution.
  • Figure 5.4: Illustration of the general collision in \ref{['sub:general_collision_for_two_balls']}. The meaning of each plot is similar to that of \ref{['fig:case3_illustration_shape']}.
  • Figure 5.5: Convergence of the iterative algorithm on the general collision example in \ref{['sub:general_collision_for_two_balls']}. The meaning of each plot is similar to that of \ref{['fig:case3_convergence_versus_iters']}.
  • ...and 10 more figures

Theorems & Definitions (6)

  • Theorem 3.1: Hybrid Minimum Principle
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5