GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping

TL;DR

This paper tackles human-assisting dexterous grasping by decomposing control into a primitive grasping gradient field learned via denoising score matching and a history-conditioned residual policy trained with PPO. GraspGF provides a direction in joint space that increases grasp likelihood conditioned on object geometry and user pose, while the residual policy modulates speed and applies corrections based on wrist trajectory history. Across a large, diverse object set and realistic human wrist trajectories, the method outperforms baselines in success, posture, and stability, and demonstrates real-world generalization without extensive fine-tuning. The approach advances practical dexterous manipulation for user-centric assistive robotics by reducing data requirements and enabling closed-loop adaptation to user intent.

Abstract

The use of anthropomorphic robotic hands for assisting individuals in situations where human hands may be unavailable or unsuitable has gained significant importance. In this paper, we propose a novel task called human-assisting dexterous grasping that aims to train a policy for controlling a robotic hand's fingers to assist users in grasping objects. Unlike conventional dexterous grasping, this task presents a more complex challenge as the policy needs to adapt to diverse user intentions, in addition to the object's geometry. We address this challenge by proposing an approach consisting of two sub-modules: a hand-object-conditional grasping primitive called Grasping Gradient Field~(GraspGF), and a history-conditional residual policy. GraspGF learns `how' to grasp by estimating the gradient from a success grasping example set, while the residual policy determines `when' and at what speed the grasping action should be executed based on the trajectory history. Experimental results demonstrate the superiority of our proposed method compared to baselines, highlighting the user-awareness and practicality in real-world applications. The codes and demonstrations can be viewed at "https://sites.google.com/view/graspgf".

Figures (13)

• Figure 1: a) Demonstration of human-assisting dexterous grasping. b) Challenges of our setting.
• Figure 2: We decompose the human-assisting dexterous grasping into learning a primitive policy $\pi_p^{\theta}$ that learns to form a pre-grasp pose and a residual policy $\pi_r^{\phi}$ that learns to adjust the proceeding of the primitive action. a) The primitive policy $\pi_p^{\theta}$ is trained on success grasping examples via score-matching objective. b) The residual policy $\pi_r^{\phi}$ is trained to adjust the primitive policy via RL.
• Figure 3: Qualitative results of comparison with baselines and different trajectories. a): final grasp poses of different methods. b): final grasp poses of our method under different human trajectories
• Figure 4: Quantitative comparative results. Left: training curve of different methods. Note that IBS takes 144 hours on V100 to reach 3.5 million agent steps, while ours only takes 15 hours to reach 10 million agent steps. Middle: Success and Posture of different methods on seen category unseen instances. Right: Success and Posture of different methods on unseen category instances.
• Figure 5: Ablation Study on decomposing policy and different action modules.