Bimanual Grasp Synthesis for Dexterous Robot Hands

TL;DR

The BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects is proposed, and the BimanGrasp-Dataset is proposed, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to the authors' knowledge.

Abstract

Humans naturally perform bimanual skills to handle large and heavy objects. To enhance robots' object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the synthesized grasps are verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose BimanGrasp-DDPM, a diffusion model trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87\% and significant acceleration in computational speed compared to BimanGrasp algorithm.

Figures (8)

• Figure 1: Our pipeline for synthesizing stable bimanual grasps, which includes: (A) generating grasp poses by initializing the bimanual grasp poses around the objects and then improving their quality through optimization; (B) verifying the grasp poses based on shape penetration and physics simulation using Isaac Gym; (C) utilizing the verified grasps (BimanGrasp-Dataset) to train a generative model (BimanGrasp-DDPM), with post-processing techniques to remove penetrations.
• Figure 2: Initialization of grasp poses (with randomization applied to joint angles and poses). We demonstrate four examples, which approach the object from different directions.
• Figure 3: Visualization of the BimanGrasp algorithm's optimization process on two objects: (A) a square container, and (B) a home appliance. Both hands started by approaching the object from initialized poses. As optimization proceeds, the hands gradually landed on object surfaces.
• Figure 4: The visualization of our physical verification of the bimanual grasps using Isaac Gym makoviychuk2021isaac. Objects are held firmly by stable grasps, while they slip away from unstable grasps.
• Figure 5: Visualization of the grasp poses synthesized on daily-life objects using BimanGrasp algorithm. Objects include: (A) backpack, (B) pot, (C) box, (D) bucket, and (E) container. All object models are from Google Scanned Dataset downs2022google.