GrainGrasp: Dexterous Grasp Generation with Fine-grained Contact Guidance

TL;DR

GrainGrasp tackles dexterous grasping by predicting distinct contact maps for each fingertip from object point clouds and optimizing grasps with a Directional Consistency optimization for Grasping (DCoG). The approach combines a CVAE-based per-finger contact-map generator trained on annotated MANO/ObMan data with a multi-term energy-based optimization that enforces finger-level contact alignment, directional consistency, and penetration avoidance, yielding controllable, human-like grasps. Ablation studies show each energy term contributes to stability and naturalness, while the per-finger design enables determinable grasps by adjusting individual finger contributions. The method demonstrates improved penetration metrics, grasp stability, and perceptual quality, albeit with reliance on CVAE generalization and the need for large-scale training data for unseen shapes.

Abstract

One goal of dexterous robotic grasping is to allow robots to handle objects with the same level of flexibility and adaptability as humans. However, it remains a challenging task to generate an optimal grasping strategy for dexterous hands, especially when it comes to delicate manipulation and accurate adjustment the desired grasping poses for objects of varying shapes and sizes. In this paper, we propose a novel dexterous grasp generation scheme called GrainGrasp that provides fine-grained contact guidance for each fingertip. In particular, we employ a generative model to predict separate contact maps for each fingertip on the object point cloud, effectively capturing the specifics of finger-object interactions. In addition, we develop a new dexterous grasping optimization algorithm that solely relies on the point cloud as input, eliminating the necessity for complete mesh information of the object. By leveraging the contact maps of different fingertips, the proposed optimization algorithm can generate precise and determinable strategies for human-like object grasping. Experimental results confirm the efficiency of the proposed scheme.

Figures (5)

• Figure 1: Examples of grasping results and grasping processes generated by the GrainGrasp. Top: The grasp results are displayed with point cloud and mesh representations in the first two rows. The colored points in the point cloud indicate the contact map of each finger. The third row illustrates the determinable results achieved by controlling the contribution of each finger. Bottom: The grasping process is represented in four steps: gray $\rightarrow$ green $\rightarrow$ yellow $\rightarrow$ blue. Throughout the process, the hand first adjusts its direction, then quickly approaches the object, which presents human-like characteristics of object grasping.
• Figure 2: Pipeline of our method. 1) We automatically annotate point cloud data and train a CVAE model to generate individual contact maps. 2) We utilize the object point cloud and contact maps to optimize the initialized hand pose. If the number of directions $d$ rotates more than once (i.e., $d > 1$), the final grasping result is obtained by minimizing $E_\text{pen}$ under the condition that $E_\text{pen} > 0$. 3) We generate determinable grasping results by adjusting individual finger contributions.
• Figure 3: (a)&(b) Five fingertips and fingers are presented in different colors for distinction. (c) Our initial hand pose exhibits a slight flexion at the interphalangeal joint compared to a fully flat hand.
• Figure 4: Qualitative experimental results. The top shows grasps generated by our complete method, the middle displays grasps obtained with the optimization-only method, and the bottom represents the Ground Truth.
• Figure 5: Determinable results. By sequentially setting the contribution of each finger to zero, we can obtain five determinable grasps.