Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper

Elizabeth Cutler, Yuning Xing, Tony Cui, Brendan Zhou, Koen van Rijnsoever, Ben Hart, David Valencia, Lee Violet C. Ong, Trevor Gee, Minas Liarokapis, Henry Williams

TL;DR

This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation in real-world contexts, demonstrating the practicality of RL training in authentic real-world scenarios.

Abstract

Reinforcement Learning (RL) training is predominantly conducted in cost-effective and controlled simulation environments. However, the transfer of these trained models to real-world tasks often presents unavoidable challenges. This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation. The benchmarking results of three RL algorithms trained on intricate in-hand manipulation tasks within practical real-world contexts are presented. Our study not only demonstrates the practicality of RL training in authentic real-world scenarios, facilitating direct real-world applications, but also provides insights into the associated challenges and considerations. Additionally, our experiences with the employed experimental methods are shared, with the aim of empowering and engaging fellow researchers and practitioners in this dynamic field of robotics.

Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper

TL;DR

This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation in real-world contexts, demonstrating the practicality of RL training in authentic real-world scenarios.

Abstract

Reinforcement Learning (RL) training is predominantly conducted in cost-effective and controlled simulation environments. However, the transfer of these trained models to real-world tasks often presents unavoidable challenges. This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation. The benchmarking results of three RL algorithms trained on intricate in-hand manipulation tasks within practical real-world contexts are presented. Our study not only demonstrates the practicality of RL training in authentic real-world scenarios, facilitating direct real-world applications, but also provides insights into the associated challenges and considerations. Additionally, our experiences with the employed experimental methods are shared, with the aim of empowering and engaging fellow researchers and practitioners in this dynamic field of robotics.
Paper Structure (21 sections, 2 equations, 7 figures, 2 tables)

This paper contains 21 sections, 2 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Gripper Design
  4. Real world RL training
  5. Dexterous Manipulator RL Benchmarks
  6. Gripper
  7. Hardware Design
  8. Physical Design
  9. Circuit Design
  10. Valve Observation Method
  11. Experimental Methods
  12. Tasks
  13. 90 degrees
  14. 90, 180, 270 degrees
  15. 30 - 330 degrees
  16. ...and 6 more sections

Figures (7)

  • Figure 1: The employed fully-actuated 9 degrees of freedom robotic gripper with the rotational sensorized target valve. A USB camera is used to detect the valve's angle and for image-based RL training. An LED ring provides stable lighting. The gripper is 3D printed and attached to an adjustable structure.
  • Figure 2: A summary of the electronic components and wiring for a single finger of the gripper
  • Figure 3: State space explanation.
  • Figure 4: Reward function calculation illustration.
  • Figure 5: The learning curves of three model-free RL algorithms, Deep Deterministic Policy Gradient (DDPG), Soft Actor Critic (SAC), and Twin Delayed DDPG (TD3) were evaluated on the three tasks. Reward at each step is determined by how much closer the gripper moves the valve towards the goal angle.
  • ...and 2 more figures