Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Goal Dexterous Hand Manipulation using Probabilistic Model-based Reinforcement Learning

Yingzhuo Jiang, Wenjun Huang, Rongdun Lin, Chenyang Miao, Tianfu Sun, Yunduan Cui

TL;DR

The paper tackles multi-goal dexterous hand manipulation with data-efficient learning by introducing GC-PMPC, a model-based RL framework that uses probabilistic neural network ensembles enhanced with Batch Normalization and a variance-penalized loss. An asynchronous MPC policy with a state-smoothing term bridges the control-frequency gap between high-fidelity models and real hardware, enabling robust, real-time control. Across four Shadow Hand simulations and a real-world DexHand 021 die-manipulation task, GC-PMPC outperforms state-of-the-art model-free and model-based baselines in learning speed, stability, and multi-goal performance, achieving complex manipulation within approximately $80$ minutes of interaction. These results demonstrate that probabilistic MBRL with MPC can enable efficient, scalable dexterous manipulation on cost-effective hardware, with potential impact on practical robotic manipulation systems.

Abstract

This paper tackles the challenge of learning multi-goal dexterous hand manipulation tasks using model-based Reinforcement Learning. We propose Goal-Conditioned Probabilistic Model Predictive Control (GC-PMPC) by designing probabilistic neural network ensembles to describe the high-dimensional dexterous hand dynamics and introducing an asynchronous MPC policy to meet the control frequency requirements in real-world dexterous hand systems. Extensive evaluations on four simulated Shadow Hand manipulation scenarios with randomly generated goals demonstrate GC-PMPC's superior performance over state-of-the-art baselines. It successfully drives a cable-driven Dexterous hand, DexHand 021 with 12 Active DOFs and 5 tactile sensors, to learn manipulating a cubic die to three goal poses within approximately 80 minutes of interactions, demonstrating exceptional learning efficiency and control performance on a cost-effective dexterous hand platform.

Multi-Goal Dexterous Hand Manipulation using Probabilistic Model-based Reinforcement Learning

TL;DR

The paper tackles multi-goal dexterous hand manipulation with data-efficient learning by introducing GC-PMPC, a model-based RL framework that uses probabilistic neural network ensembles enhanced with Batch Normalization and a variance-penalized loss. An asynchronous MPC policy with a state-smoothing term bridges the control-frequency gap between high-fidelity models and real hardware, enabling robust, real-time control. Across four Shadow Hand simulations and a real-world DexHand 021 die-manipulation task, GC-PMPC outperforms state-of-the-art model-free and model-based baselines in learning speed, stability, and multi-goal performance, achieving complex manipulation within approximately minutes of interaction. These results demonstrate that probabilistic MBRL with MPC can enable efficient, scalable dexterous manipulation on cost-effective hardware, with potential impact on practical robotic manipulation systems.

Abstract

This paper tackles the challenge of learning multi-goal dexterous hand manipulation tasks using model-based Reinforcement Learning. We propose Goal-Conditioned Probabilistic Model Predictive Control (GC-PMPC) by designing probabilistic neural network ensembles to describe the high-dimensional dexterous hand dynamics and introducing an asynchronous MPC policy to meet the control frequency requirements in real-world dexterous hand systems. Extensive evaluations on four simulated Shadow Hand manipulation scenarios with randomly generated goals demonstrate GC-PMPC's superior performance over state-of-the-art baselines. It successfully drives a cable-driven Dexterous hand, DexHand 021 with 12 Active DOFs and 5 tactile sensors, to learn manipulating a cubic die to three goal poses within approximately 80 minutes of interactions, demonstrating exceptional learning efficiency and control performance on a cost-effective dexterous hand platform.
Paper Structure (14 sections, 5 equations, 10 figures)

This paper contains 14 sections, 5 equations, 10 figures.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Model-based Reinforcement Learning
  4. Dexterous Hand Manipulation Scenarios
  5. Approach
  6. Goal-Conditioned Neural Network Ensembles
  7. Asynchronous MPC Policy with State Smoothing
  8. Overview of GC-PMPC
  9. Experimental Results
  10. Experimental Settings
  11. Simulation Results
  12. Real-world Hardware Results
  13. Engineering Challenges in Real-World Dexterous Hand
  14. Conclusions

Figures (10)

  • Figure 1: Real-world dexterous hand used in this work. It was successfully drove to learn a multi-goal die rotation task within $80$ minutes based on a single-camera pose detection.
  • Figure 2: Manipulation Scenarios on simulated and real-world dexterous hand systems.
  • Figure 3: Principles of the asynchronous MPC policy ($x=2$) designed to match the control frequency of real-world dexterous hand systems.
  • Figure 4: Workflow of GC-PMPC on the real-world DexHand 021 System.
  • Figure 5: Learning curves of GC-PMPC and other baselines in simulated manipulation tasks. The shaded region represents the corresponding standard deviation.
  • ...and 5 more figures