Dexterous grasp data augmentation based on grasp synthesis with fingertip workspace cloud and contact-aware sampling

Abstract

Robotic grasping is a fundamental yet crucial component of robotic applications, as effective grasping often serves as the starting point for various tasks. With the rapid advancement of neural networks, data-driven approaches for robotic grasping have become mainstream. However, efficiently generating grasp datasets for training remains a bottleneck. This is compounded by the diverse structures of robotic hands, making the design of generalizable grasp generation methods even more complex. In this work, we propose a teleoperation-based framework to collect a small set of grasp pose demonstrations, which are augmented using FSG--a Fingertip-contact-aware Sampling-based Grasp generator. Based on the demonstrated grasp poses, we propose AutoWS, which automatically generates structured workspace clouds of robotic fingertips, embedding the hand structure information directly into the clouds to eliminate the need for inverse kinematics calculations. Experiments on grasping the YCB objects show that our method significantly outperforms existing approaches in both speed and valid pose generation rate. Our framework enables real-time grasp generation for hands with arbitrary structures and produces human-like grasps when combined with demonstrations, providing an efficient and robust data augmentation tool for data-driven grasp training.

Figures (9)

• Figure 1: System architecture for dexterous grasp data augmentation. The general grasp generator consists of two components: AutoWS and FSG. Given a robot hand model, AutoWS automatically generates the fingertip workspace clouds. Based on the generated workspace clouds and the shape information of the target object, FSG rapidly generates valid grasp poses through fingertip-contact-aware random sampling. When a human-demonstrated grasp pose is provided, AutoWS generates the workspace clouds corresponding to the demonstrated pose, and FSG generates grasp data with finger joint angles similar to the demonstration but with different grasp locations, enabling grasp data augmentation.
• Figure 2: Flowchart of AutoWS. (a) Starting with a robot hand model, (b) the distal link cloud is sampled using PDS and preprocessed based on two vectors, $v_{u}$(upward) and $v_{f}$(forward), in $frame_{tip}$. (c-d) Combined with a demonstrated grasp, (e-g) contact faces are approximated. Using the hand structure and joint space information, (h-i) fingertip workspace clouds in $frame_{palm}$ are generated and filtered, along with (j) corresponding finger skeleton clouds. Each point in the workspace cloud represents a potential contact point for the fingertip, while the line segments between point pairs in the skeleton clouds represents the finger links, excluding fingertips, at specific joint angles.
• Figure 3: Flowchart of FSG. Fingertip workspace clouds and object clouds are inputs to FSG. (a) Temporarily align $frame_{obj}$ with $frame_{palm}$. (b) Generate palm pose candidates by random sampling and matching contact point pairs between $cloud_{part}$ and the workspace cloud with least points. (c) Reconstruct fingertip contact face. (d) Find contact points from $cloud_{full}$ located within fingertip contact face. (e) Filter out neighbor points from $cloud_{part}$. (f-h) Iteratively search for contact faces and points of other fingers. (i) Final contact points from $cloud_{full}$ located within all fingertip contact faces. (j) Get hand skeleton clouds based on final contact points. (k) Generated grasp pose for further quality evaluation.
• Figure 4: Poses generated for the Shadow Hand to grasp the ycb 021-bleach-cleanser with different number of fingers.
• Figure 5: Grasp poses on concave surface without link penetration checking.