Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DexGrasp-Diffusion: Diffusion-based Unified Functional Grasp Synthesis Method for Multi-Dexterous Robotic Hands

Zhengshen Zhang, Lei Zhou, Chenchen Liu, Zhiyang Liu, Chengran Yuan, Sheng Guo, Ruiteng Zhao, Marcelo H. Ang, Francis EH Tay

TL;DR

The capacity of DexGrasp-Diffusion to reliably generate functional grasps for household objects aligned with specific affordance instructions is demonstrated, supporting the superior performance of MultiHandDiffuser over the baseline model in terms of success rate, grasp diversity, and collision depth.

Abstract

The versatility and adaptability of human grasping catalyze advancing dexterous robotic manipulation. While significant strides have been made in dexterous grasp generation, current research endeavors pivot towards optimizing object manipulation while ensuring functional integrity, emphasizing the synthesis of functional grasps following desired affordance instructions. This paper addresses the challenge of synthesizing functional grasps tailored to diverse dexterous robotic hands by proposing DexGrasp-Diffusion, an end-to-end modularized diffusion-based method. DexGrasp-Diffusion integrates MultiHandDiffuser, a novel unified data-driven diffusion model for multi-dexterous hands grasp estimation, with DexDiscriminator, which employs a Physics Discriminator and a Functional Discriminator with open-vocabulary setting to filter physically plausible functional grasps based on object affordances. The experimental evaluation conducted on the MultiDex dataset provides substantiating evidence supporting the superior performance of MultiHandDiffuser over the baseline model in terms of success rate, grasp diversity, and collision depth. Moreover, we demonstrate the capacity of DexGrasp-Diffusion to reliably generate functional grasps for household objects aligned with specific affordance instructions.

DexGrasp-Diffusion: Diffusion-based Unified Functional Grasp Synthesis Method for Multi-Dexterous Robotic Hands

TL;DR

The capacity of DexGrasp-Diffusion to reliably generate functional grasps for household objects aligned with specific affordance instructions is demonstrated, supporting the superior performance of MultiHandDiffuser over the baseline model in terms of success rate, grasp diversity, and collision depth.

Abstract

The versatility and adaptability of human grasping catalyze advancing dexterous robotic manipulation. While significant strides have been made in dexterous grasp generation, current research endeavors pivot towards optimizing object manipulation while ensuring functional integrity, emphasizing the synthesis of functional grasps following desired affordance instructions. This paper addresses the challenge of synthesizing functional grasps tailored to diverse dexterous robotic hands by proposing DexGrasp-Diffusion, an end-to-end modularized diffusion-based method. DexGrasp-Diffusion integrates MultiHandDiffuser, a novel unified data-driven diffusion model for multi-dexterous hands grasp estimation, with DexDiscriminator, which employs a Physics Discriminator and a Functional Discriminator with open-vocabulary setting to filter physically plausible functional grasps based on object affordances. The experimental evaluation conducted on the MultiDex dataset provides substantiating evidence supporting the superior performance of MultiHandDiffuser over the baseline model in terms of success rate, grasp diversity, and collision depth. Moreover, we demonstrate the capacity of DexGrasp-Diffusion to reliably generate functional grasps for household objects aligned with specific affordance instructions.
Paper Structure (18 sections, 4 equations, 5 figures, 2 tables)

This paper contains 18 sections, 4 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Data-Driven Dexterous Grasping
  4. Diffusion Models for Robotics
  5. Method
  6. Problem Statement
  7. MultiHandDiffuser
  8. Physics Discriminator
  9. Functional Discriminator
  10. Experiments
  11. Dataset
  12. Evaluation Metrics
  13. Implementation Details
  14. Results and Analysis
  15. Quantitative Analysis
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Overview of the diffusion sampling process with fixed target objects for multi-dexterous robotic hands performed by our presented DexGrasp-Diffusion method. (a) EZGripper with apple. (b) Barrett with power drill. (c) Robotiq-3F with camera. (d) Allegro with water bottle. (e) ShadowHand with wine glass.
  • Figure 2: Overview of our proposed DexGrasp-Diffusion method.
  • Figure 3: Generated physically plausible grasp candidates for unseen objects. (a) EZGripper. (b) Barrett. (c) Allegro. (d) ShadowHand.
  • Figure 4: Qualitative results of detected functional grasps by DexGrasp-Diffusion. The unseen affordances are shown in orange.
  • Figure 5: Some failure or counter-intuitive cases of our method. The unseen affordances are shown in orange.