Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Adaptive Framework for Manipulator Skill Reproduction in Dynamic Environments

Ryan Donald, Brendan Hertel, Stephen Misenti, Yan Gu, Reza Azadeh

TL;DR

The paper tackles manipulation in dynamic environments by introducing a three-component framework that blends ELTE-based Learning from Demonstration for online trajectory adaptation, UKF-driven environment state prediction for proactive planning, and HMM-based high-level decision making with an FSM for safety. ELTE optimizes a convex energy f0(y)=U_Y+U_E+U_R to deform trajectories online while preserving demonstrated shapes, including online endpoint updates; UKF uses a constant-acceleration model to predict future states and update trajectories; the HMM governs safe execution by transitioning between Forward, Pause, and Reverse states, triggering safety actions as needed. Validation includes simulation and 36 real-world runs on a legged mobile manipulator, demonstrating robustness, improved tracking, and safer execution under dynamic perturbations. The work highlights meaningful advancements in proactive adaptation for manipulation tasks and provides a concrete framework applicable to dynamic and uncertain environments with potential for broader deployment in autonomous robotics.

Abstract

Robot skill learning and execution in uncertain and dynamic environments is a challenging task. This paper proposes an adaptive framework that combines Learning from Demonstration (LfD), environment state prediction, and high-level decision making. Proactive adaptation prevents the need for reactive adaptation, which lags behind changes in the environment rather than anticipating them. We propose a novel LfD representation, Elastic-Laplacian Trajectory Editing (ELTE), which continuously adapts the trajectory shape to predictions of future states. Then, a high-level reactive system using an Unscented Kalman Filter (UKF) and Hidden Markov Model (HMM) prevents unsafe execution in the current state of the dynamic environment based on a discrete set of decisions. We first validate our LfD representation in simulation, then experimentally assess the entire framework using a legged mobile manipulator in 36 real-world scenarios. We show the effectiveness of the proposed framework under different dynamic changes in the environment. Our results show that the proposed framework produces robust and stable adaptive behaviors.

An Adaptive Framework for Manipulator Skill Reproduction in Dynamic Environments

TL;DR

The paper tackles manipulation in dynamic environments by introducing a three-component framework that blends ELTE-based Learning from Demonstration for online trajectory adaptation, UKF-driven environment state prediction for proactive planning, and HMM-based high-level decision making with an FSM for safety. ELTE optimizes a convex energy f0(y)=U_Y+U_E+U_R to deform trajectories online while preserving demonstrated shapes, including online endpoint updates; UKF uses a constant-acceleration model to predict future states and update trajectories; the HMM governs safe execution by transitioning between Forward, Pause, and Reverse states, triggering safety actions as needed. Validation includes simulation and 36 real-world runs on a legged mobile manipulator, demonstrating robustness, improved tracking, and safer execution under dynamic perturbations. The work highlights meaningful advancements in proactive adaptation for manipulation tasks and provides a concrete framework applicable to dynamic and uncertain environments with potential for broader deployment in autonomous robotics.

Abstract

Robot skill learning and execution in uncertain and dynamic environments is a challenging task. This paper proposes an adaptive framework that combines Learning from Demonstration (LfD), environment state prediction, and high-level decision making. Proactive adaptation prevents the need for reactive adaptation, which lags behind changes in the environment rather than anticipating them. We propose a novel LfD representation, Elastic-Laplacian Trajectory Editing (ELTE), which continuously adapts the trajectory shape to predictions of future states. Then, a high-level reactive system using an Unscented Kalman Filter (UKF) and Hidden Markov Model (HMM) prevents unsafe execution in the current state of the dynamic environment based on a discrete set of decisions. We first validate our LfD representation in simulation, then experimentally assess the entire framework using a legged mobile manipulator in 36 real-world scenarios. We show the effectiveness of the proposed framework under different dynamic changes in the environment. Our results show that the proposed framework produces robust and stable adaptive behaviors.
Paper Structure (11 sections, 3 equations, 6 figures, 2 tables)

This paper contains 11 sections, 3 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Adaptive Skill Learning
  5. Environment Prediction
  6. Reactive System and High-Level Decision Making
  7. Experiments
  8. Validation of the Adaptive Skill Learning in Simulation
  9. Experimental Setup
  10. Real-world Experiments
  11. Conclusions and Future Work

Figures (6)

  • Figure 1: Experimental setup with Kinova Jaco2 mounted on a Ghost Robotics Vision 60.
  • Figure 2: An overview of all components of the proposed adaptive skill learning and execution framework.
  • Figure 3: Finite State Machine (FSM) that controls high-level state transition based on changes in the environment.
  • Figure 4: (Left) An example of optimal continuations found for a given execution of a 2D reaching trajectory. The reproduction must smoothly approach a given perturbed goal while avoiding the obstacle. (Middle) An example of optimal adaptive continuations found for a given execution of a 2D reaching trajectory. The reproduction smoothly changes with the changing endpoint. (Right) A comparison of ELTE and DMPs for an endpoint changing partially through execution.
  • Figure 5: Execution of an inspection task before (left) and after (center) the environment changes and the target is moved. Our framework reacts appropriately to the change in the environment and successfully executes the task (right).
  • ...and 1 more figures