GenDexGrasp: Generalizable Dexterous Grasping

TL;DR

GenDexGrasp tackles the challenge of generalizable dexterous grasping across unseen hands by introducing a hand-agnostic contact-map representation and an efficient optimization pipeline. It learns a CVAE to generate object-centric contact maps and fits any unseen hand to these maps through differentiable grasp optimization, followed by a physics-based refinement; a novel aligned distance improves contact accuracy on thin objects. The approach is trained on MultiDex, a large synthetic dataset of 436k grasps across 5 hands and 58 objects, enabling robust generalization and rapid inference. Empirically, GenDexGrasp achieves a favorable three-way trade-off among speed, diversity, and generalizability, outperforming prior hand-agnostic methods in efficiency and prior hand-aware methods in diversity, with practical implications for rapid prototyping of new robotic hands.

Abstract

Generating dexterous grasping has been a long-standing and challenging robotic task. Despite recent progress, existing methods primarily suffer from two issues. First, most prior arts focus on a specific type of robot hand, lacking the generalizable capability of handling unseen ones. Second, prior arts oftentimes fail to rapidly generate diverse grasps with a high success rate. To jointly tackle these challenges with a unified solution, we propose GenDexGrasp, a novel hand-agnostic grasping algorithm for generalizable grasping. GenDexGrasp is trained on our proposed large-scale multi-hand grasping dataset MultiDex synthesized with force closure optimization. By leveraging the contact map as a hand-agnostic intermediate representation, GenDexGrasp efficiently generates diverse and plausible grasping poses with a high success rate and can transfer among diverse multi-fingered robotic hands. Compared with previous methods, GenDexGrasp achieves a three-way trade-off among success rate, inference speed, and diversity. Code is available at https://github.com/tengyu-liu/GenDexGrasp.

Figures (6)

• Figure 1: Exemplar grasps of different hands and objects from the proposed synthesized dataset. From top to bottom: EZGripper, Barrett, Robotiq-3F, Allegro, and ShadowHand. From left to right: alarm clock, apple, binocular, and meat can.
• Figure 2: Comparison between aligned and euclidean distances on thin shell objects. Given an exemplar grasp (a), we show both distances from the object to hand surfaces in 3D; red regions denote shorter distances and blue longer. An illustration of both distances is also shown in 2D (b,c); the green rectangle, white cross, and green arrow represent a rectangular object, the point of interest, and the surface normal $n_o$ at the point, respectively. The Euclidean distance (b) labels the upper edge of the object as close to the point of interest, whereas the aligned distance (c) is geometry-aware. The 3D aligned distances of the exemplar grasp (e) correctly reflect non-contact areas in the highlighted area, where the finger contacts the opposite side of the thin object. The Euclidean distances (d) fail to distinguish contacts on one side from contacts on the other side.
• Figure 3: An overview of the GenDexGrasp pipeline. We first collect a large-scale synthetic dataset for multiple hands with dfc. Then, we train a CVAE to generate hand-agnostic contact maps for unseen objects. We finally optimize grasping poses for unseen hands using the generated contact maps.
• Figure 4: Examples of the generated grasping poses for unseen hands and objects. From top to bottom: Barrett, Allegro, and ShadowHand.
• Figure 5: Failure cases with Allegro (top) and ShadowHand (bottom). The last column shows artifacts caused by contact ambiguities when using Euclidean distances instead of aligned distances.