Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An NMPC-ECBF Framework for Dynamic Motion Planning and Execution in vision-based Human-Robot Collaboration

Dianhao Zhang, Mien Van, Pantelis Sopasakis, Seán McLoone

TL;DR

A novel safety-critical paradigm that uses an exponential control barrier function (ECBF) as a safety filter is formulated and a 23.2% reduction in execution time is achieved for the HRC task compared to an implementation without human motion prediction.

Abstract

To enable safe and effective human-robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition, and prediction into the robot controller is critical for real-time awareness, response, and communication inside a heterogeneous environment (robots, humans, and equipment). The proposed approach takes advantage of the prediction capabilities of nonlinear model predictive control (NMPC) to execute a safe path planning based on feedback from a vision system. In order to satisfy the requirement of real-time path planning, an embedded solver based on a penalty method is applied. However, due to tight sampling times NMPC solutions are approximate, and hence the safety of the system cannot be guaranteed. To address this we formulate a novel safety-critical paradigm with an exponential control barrier function (ECBF) used as a safety filter. We also design a simple human-robot collaboration scenario using V-REP to evaluate the performance of the proposed controller and investigate whether integrating human pose prediction can help with safe and efficient collaboration. The robot uses OptiTrack cameras for perception and dynamically generates collision-free trajectories to the predicted target interactive position. Results for a number of different configurations confirm the efficiency of the proposed motion planning and execution framework. It yields a 19.8% reduction in execution time for the HRC task considered.

An NMPC-ECBF Framework for Dynamic Motion Planning and Execution in vision-based Human-Robot Collaboration

TL;DR

A novel safety-critical paradigm that uses an exponential control barrier function (ECBF) as a safety filter is formulated and a 23.2% reduction in execution time is achieved for the HRC task compared to an implementation without human motion prediction.

Abstract

To enable safe and effective human-robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition, and prediction into the robot controller is critical for real-time awareness, response, and communication inside a heterogeneous environment (robots, humans, and equipment). The proposed approach takes advantage of the prediction capabilities of nonlinear model predictive control (NMPC) to execute a safe path planning based on feedback from a vision system. In order to satisfy the requirement of real-time path planning, an embedded solver based on a penalty method is applied. However, due to tight sampling times NMPC solutions are approximate, and hence the safety of the system cannot be guaranteed. To address this we formulate a novel safety-critical paradigm with an exponential control barrier function (ECBF) used as a safety filter. We also design a simple human-robot collaboration scenario using V-REP to evaluate the performance of the proposed controller and investigate whether integrating human pose prediction can help with safe and efficient collaboration. The robot uses OptiTrack cameras for perception and dynamically generates collision-free trajectories to the predicted target interactive position. Results for a number of different configurations confirm the efficiency of the proposed motion planning and execution framework. It yields a 19.8% reduction in execution time for the HRC task considered.
Paper Structure (23 sections, 36 equations, 7 figures, 3 tables, 2 algorithms)

This paper contains 23 sections, 36 equations, 7 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Overall System Design
  3. Related Works
  4. Methods to solve trajectory optimization in MPC
  5. Safety-critical Manipulator Control
  6. Problem Formulation
  7. NMPC Motion Planning Scheme
  8. System Description
  9. Minimum distance calculation
  10. Nonlinear model predictive control for collision avoidance
  11. Numerical solution
  12. Controller Design
  13. Trajectory Computation
  14. Low-level controller
  15. Exponential Control Barrier Function as a Safety Filter
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Proposed human-robot collaboration system design
  • Figure 2: Work flow from non-local graph convolution (NGC) based vision system to path planning. The outputs from the recognition and prediction modules are estimated human action category and future pose sequence, respectively.
  • Figure 3: Calculation of the distance, position and orientation of the capsules for the representation of humans.
  • Figure 4: NMPC-based motion planning system
  • Figure 5: Confusion matrix describing the accuracy of action recognition in the screw-driver usage task.
  • ...and 2 more figures