FunGrasp: Functional Grasping for Diverse Dexterous Hands

TL;DR

FunGrasp is introduced, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands.

Abstract

Functional grasping is essential for humans to perform specific tasks, such as grasping scissors by the finger holes to cut materials or by the blade to safely hand them over. Enabling dexterous robot hands with functional grasping capabilities is crucial for their deployment to accomplish diverse real-world tasks. Recent research in dexterous grasping, however, often focuses on power grasps while overlooking task- and object-specific functional grasping poses. In this paper, we introduce FunGrasp, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects. Given a single RGBD image of functional human grasping, our system estimates the hand pose and transfers it to different robotic hands via a human-to-robot (H2R) grasp retargeting module. Guided by the retargeted grasping poses, a policy is trained through reinforcement learning in simulation for dynamic grasping control. To achieve robust sim-to-real transfer, we employ several techniques including privileged learning, system identification, domain randomization, and gravity compensation. In our experiments, we demonstrate that our system enables diverse functional grasping of unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands. The significance of the components is validated through comprehensive ablation studies. Project page: https://hly-123.github.io/FunGrasp/ .

Figures (6)

• Figure 2: Our method achieves task-specific functional dexterous grasping for different robot hands with single human grasp RGBD images as input.
• Figure 3: System Overview. Our system contains three modules: A) H2R Grasp Retargeting, which retargets the functional grasp poses from the human hand to diverse robot hands. B) Dynamic Grasp Control, which controls the robot hands to achieve dynamic grasping motions with a policy trained by RL. C) Sim-to-Real Transfer, which applies techniques including privileged learning and system identification for robust hardware deployment.
• Figure 4: Hardware setup.
• Figure 5: Objects used for one-shot generalization evaluation.
• Figure 6: Diverse functional grasps from single RGBD images.