Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

Mehdi Heydari Shahna, Seyed Adel Alizadeh Kolagar, Jouni Mattila

TL;DR

This work proposes integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy, all while actively engaging in the learning phase through interactions with the environment.

Abstract

In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability, which may pose challenges in ensuring stability and safety. To address these issues, we propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy, all while actively engaging in the learning phase through interactions with the environment. This approach circumvents the control performance and complexities associated with computations while addressing nonrepetitive reaching tasks in the presence of obstacles. First, a model-free DRL agent is employed to plan velocity-bounded motion for a manipulator with 'n' degrees of freedom (DoF), ensuring collision avoidance for the end-effector through joint-level reasoning. The generated reference motion is then input into a robust subsystem-based adaptive controller, which produces the necessary torques, while the cuckoo search optimization (CSO) algorithm enhances control gains to minimize the stabilization and tracking error in the steady state. This approach guarantees robustness and uniform exponential convergence in an unfamiliar environment, despite the presence of uncertainties and disturbances. Theoretical assertions are validated through the presentation of simulation outcomes.

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

TL;DR

This work proposes integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy, all while actively engaging in the learning phase through interactions with the environment.

Abstract

In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability, which may pose challenges in ensuring stability and safety. To address these issues, we propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy, all while actively engaging in the learning phase through interactions with the environment. This approach circumvents the control performance and complexities associated with computations while addressing nonrepetitive reaching tasks in the presence of obstacles. First, a model-free DRL agent is employed to plan velocity-bounded motion for a manipulator with 'n' degrees of freedom (DoF), ensuring collision avoidance for the end-effector through joint-level reasoning. The generated reference motion is then input into a robust subsystem-based adaptive controller, which produces the necessary torques, while the cuckoo search optimization (CSO) algorithm enhances control gains to minimize the stabilization and tracking error in the steady state. This approach guarantees robustness and uniform exponential convergence in an unfamiliar environment, despite the presence of uncertainties and disturbances. Theoretical assertions are validated through the presentation of simulation outcomes.
Paper Structure (12 sections, 40 equations, 6 figures, 3 tables)

This paper contains 12 sections, 40 equations, 6 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. Modeling an n-DoF Robotic Manipulator System
  3. Reinforcement Learning-based Trajectory Planning
  4. State and Action Representation
  5. Reward Function
  6. Learning Procedure
  7. ROBUST DRL-BASED CONTROL STRUCTURE
  8. Cuckoo Search Optimization
  9. Adaptive subsystem-based control
  10. Stability analysis
  11. Numerical Evaluation
  12. Conclusion

Figures (6)

  • Figure 1: The architecture of RL-based learning liu2022robot.
  • Figure 2: Integration of the low-level robust subsystem-based control with collision-free DRL-based motion planner.
  • Figure 3: Manipulator's tip tracking collision-free trajectories for non-repetitive tasks in different views.
  • Figure 4: The position target error in task space.
  • Figure 5: Torques generated by joints to perform control targets.
  • ...and 1 more figures