Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing

Siddharth Singh, Tian Yu, Qing Chang, John Karigiannis, Shaopeng Liu

TL;DR

This work tackles automatic robot motion planning in dynamic manufacturing settings by proposing a multi-level hybrid framework that unifies a task-space RL-LfD agent with a joint-space DRL planner, coordinated by an RL-based switching policy. A feasibility map that integrates reachability, manipulability, and collision checks guides trajectory generation, enabling efficient offline training and robust online execution. The RL-LfD component quickly captures task-space constraints from demonstrations, while the DRL component corrects infeasible joint-space segments, with the switching policy ensuring smooth transitions. Across simulation and industrial case studies, the hybrid approach outperforms pure DRL or LfD methods in training efficiency and task success, demonstrating practical potential for adaptive, collision-free motion in condensed workspaces with humans and machines.

Abstract

Industrial robots are widely used in diverse manufacturing environments. Nonetheless, how to enable robots to automatically plan trajectories for changing tasks presents a considerable challenge. Further complexities arise when robots operate within work cells alongside machines, humans, or other robots. This paper introduces a multi-level hybrid robot motion planning method combining a task space Reinforcement Learning-based Learning from Demonstration (RL-LfD) agent and a joint-space based Deep Reinforcement Learning (DRL) based agent. A higher level agent learns to switch between the two agents to enable feasible and smooth motion. The feasibility is computed by incorporating reachability, joint limits, manipulability, and collision risks of the robot in the given environment. Therefore, the derived hybrid motion planning policy generates a feasible trajectory that adheres to task constraints. The effectiveness of the method is validated through sim ulated robotic scenarios and in a real-world setup.

Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing

TL;DR

This work tackles automatic robot motion planning in dynamic manufacturing settings by proposing a multi-level hybrid framework that unifies a task-space RL-LfD agent with a joint-space DRL planner, coordinated by an RL-based switching policy. A feasibility map that integrates reachability, manipulability, and collision checks guides trajectory generation, enabling efficient offline training and robust online execution. The RL-LfD component quickly captures task-space constraints from demonstrations, while the DRL component corrects infeasible joint-space segments, with the switching policy ensuring smooth transitions. Across simulation and industrial case studies, the hybrid approach outperforms pure DRL or LfD methods in training efficiency and task success, demonstrating practical potential for adaptive, collision-free motion in condensed workspaces with humans and machines.

Abstract

Industrial robots are widely used in diverse manufacturing environments. Nonetheless, how to enable robots to automatically plan trajectories for changing tasks presents a considerable challenge. Further complexities arise when robots operate within work cells alongside machines, humans, or other robots. This paper introduces a multi-level hybrid robot motion planning method combining a task space Reinforcement Learning-based Learning from Demonstration (RL-LfD) agent and a joint-space based Deep Reinforcement Learning (DRL) based agent. A higher level agent learns to switch between the two agents to enable feasible and smooth motion. The feasibility is computed by incorporating reachability, joint limits, manipulability, and collision risks of the robot in the given environment. Therefore, the derived hybrid motion planning policy generates a feasible trajectory that adheres to task constraints. The effectiveness of the method is validated through sim ulated robotic scenarios and in a real-world setup.

Paper Structure

This paper contains 24 sections, 23 equations, 11 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Mathematical Background
  3. Mathematical Notations
  4. Dual Quaternion
  5. Markov Decision Process Formulation
  6. Hybrid Motion Planning Framework
  7. Problem Formulation
  8. Proposed Framework
  9. Joint Space Feasibility Analysis
  10. Integrated Feasibility Map
  11. Hybrid Motion Planning with RL based Switching Agent
  12. Task-Space RL based LfD
  13. Joint-Space Feasibility DRL Motion Planner
  14. MDP Formulation
  15. Online Execution of the Joint Space Motion Planner
  16. ...and 9 more sections

Figures (11)

  • Figure 1: Framework of the proposed multi-level hybrid motion planning technique.
  • Figure 2: Schematic picture of the feasibility map in a robotic workspace.
  • Figure 3: Mapping the demonstrated skill (blue line) to the new task with the new starting and goal configuration $D_1'$ and $D_n'$.
  • Figure 4: Human-in-the-loop LfD based RL scheme
  • Figure 5: Experimental setup with feasibility map used for simulation study.
  • ...and 6 more figures