Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-time Model Predictive Control with Zonotope-Based Neural Networks for Bipedal Social Navigation

Abdulaziz Shamsah, Krishanu Agarwal, Shreyas Kousik, Ye Zhao

TL;DR

Two cascaded zonotope-based neural networks are proposed: a Pedestrian Prediction Network (PPN) for pedestrians’ future trajectory prediction and an Ego-agent Social Network (ESN) for ego-agent social path planning.

Abstract

This study addresses the challenge of bipedal navigation in a dynamic human-crowded environment, a research area that remains largely underexplored in the field of legged navigation. We propose two cascaded zonotope-based neural networks: a Pedestrian Prediction Network (PPN) for pedestrians' future trajectory prediction and an Ego-agent Social Network (ESN) for ego-agent social path planning. Representing future paths as zonotopes allows for efficient reachability-based planning and collision checking. The ESN is then integrated with a Model Predictive Controller (ESN-MPC) for footstep planning for our bipedal robot Digit designed by Agility Robotics. ESN-MPC solves for a collision-free optimal trajectory by optimizing through the gradients of ESN. ESN-MPC optimal trajectory is sent to the low-level controller for full-order simulation of Digit. The overall proposed framework is validated with extensive simulations on randomly generated initial settings with varying human crowd densities.

Real-time Model Predictive Control with Zonotope-Based Neural Networks for Bipedal Social Navigation

TL;DR

Two cascaded zonotope-based neural networks are proposed: a Pedestrian Prediction Network (PPN) for pedestrians’ future trajectory prediction and an Ego-agent Social Network (ESN) for ego-agent social path planning.

Abstract

This study addresses the challenge of bipedal navigation in a dynamic human-crowded environment, a research area that remains largely underexplored in the field of legged navigation. We propose two cascaded zonotope-based neural networks: a Pedestrian Prediction Network (PPN) for pedestrians' future trajectory prediction and an Ego-agent Social Network (ESN) for ego-agent social path planning. Representing future paths as zonotopes allows for efficient reachability-based planning and collision checking. The ESN is then integrated with a Model Predictive Controller (ESN-MPC) for footstep planning for our bipedal robot Digit designed by Agility Robotics. ESN-MPC solves for a collision-free optimal trajectory by optimizing through the gradients of ESN. ESN-MPC optimal trajectory is sent to the low-level controller for full-order simulation of Digit. The overall proposed framework is validated with extensive simulations on randomly generated initial settings with varying human crowd densities.
Paper Structure (27 sections, 2 theorems, 14 equations, 9 figures, 2 tables)

This paper contains 27 sections, 2 theorems, 14 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem formulation
  4. Robot Model
  5. Environment Setup and Problem Statement
  6. Preliminaries
  7. Environment Assumptions and Observations
  8. Zonotopes Preliminaries
  9. Social Zonotope Network
  10. Learning Architecture
  11. Pedestrian Prediction Network (PPN)
  12. Ego-agent Social Network (ESN)
  13. Zonotope Shaping Loss Functions
  14. Social MPC
  15. Kinematics Constraints
  16. ...and 12 more sections

Key Result

Proposition IV.2

(guibas2003zonotopes) $\mathcal{Z}_1 \cap \mathcal{Z}_1 = \emptyset$ iff $\boldsymbol{c}_1 \notin {{\mathscr{Z}\!}\!\left(\boldsymbol{c}_2, [G_1 \; G_2]\right)}$.

Figures (9)

  • Figure 1: Snapshot of the simulation environment with superimposed zonotopes for the proposed reachability-based social navigation framework. The environment is a $14$ m $\times$$14$ m open space with $20$ pedestrians.
  • Figure 2: Block diagram of the proposed framework. The framework is composed of two sub-networks: the Pedestrian Prediction Network (PPN) and the Ego-agent Social Network (ESN) shown in green and cyan, respectively (Sec. \ref{['sec:social_zono_net']}). Given an environment with observed pedestrians and a goal location, PPN predicts the future pedestrians' reachable set. ESN-MPC optimizes through ESN to generate collision-free trajectories for Digit (Sec. \ref{['sec:SDMPC']}). The optimal trajectory is then sent to the ALIP controller Gong2022AngularMomentum to generate the desired foot placement for reduced-order optimal trajectory tracking. An ankle-actuated-passivity-based controller sadeghian2017passivityshamsah2023integrated is implemented on Digit for full-body trajectory tracking.
  • Figure 3: Illustration of the Linear Inverted Pendulum model for two consecutive walking steps, with discrete states $\boldsymbol{p} _{q}$ and $\boldsymbol{p} _{q+1}$ at the contact switching time. The shaded yellow regions indicate the kinematics constraint on the control input $\boldsymbol{u}$ detailed in Sec. \ref{['subsec:kin_const']}.
  • Figure 4: (a) shows the pedestrian prediction network, conditioned on the pedestrian endpoints and the immediate change in the ego-agent's state. (b) shows the ego-agent social network conditioned on the pedestrians' future prediction, the immediate change in the ego-agent's state, and the ego-agent's goal location. Dashed connections are used during training only.
  • Figure 5: Our zonotope shaping loss functions. The loss aims to learn interconnected zonotopes that engulf the ground truth path.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Definition III.1: Navigation safety
  • Definition III.2: Socially acceptable path for bipedal systems
  • Proposition IV.2
  • Proposition IV.3
  • Definition IV.1: Social Zonotope $\mathcal{Z} ^{\rm ego} _{q}$
  • Remark 1