Cross-Category Functional Grasp Transfer

TL;DR

This work analyzes and collects grasp-related similarity relationships among 51 common tool-like object categories and annotates semantic grasp representation for 1768 objects to infer a cross-category functional grasp synthesis.

Abstract

Generating grasps for a dexterous hand often requires numerous grasping annotations. However, annotating high DoF dexterous hand poses is quite challenging. Especially for functional grasps, requiring the hand to grasp the object in a specific pose to facilitate subsequent manipulations. This prompts us to explore how people achieve manipulations on new objects based on past grasp experiences. We find that when grasping new items, people are adept at discovering and leveraging various similarities between objects, including shape, layout, and grasp type. Considering this, we analyze and collect grasp-related similarity relationships among 51 common tool-like object categories and annotate semantic grasp representation for 1768 objects. These objects are connected through similarities to form a knowledge graph, which helps infer our proposed cross-category functional grasp synthesis. Through extensive experiments, we demonstrate that the grasp-related knowledge indeed contributed to achieving functional grasp transfer across unknown or entirely new categories of objects.

Figures (9)

• Figure 1: (a) Three object attributes related to cross-category grasp transfer, as well as representative objects. (b) The part of knowledge graph ontology constructed based on the similarity of objects' three attributes. (c) Presentation of some objects in the dataset.
• Figure 2: The nine functional grasp types named after representative objects, each shows two examples.
• Figure 3: (a) An example of functional grasp. (b) The 16 parts of human hands. (c) A example of the touch code.
• Figure 4: (a) Training stage of our prediction framework. (b) Inference stage for new objects.
• Figure 5: The auto-encoder network is designed to obtain vector-form features of the object point cloud or the object point cloud annotated with touch code. 'n' is the number of object's surface points, and 'd' is the feature dimension of each point. When the input is only the object's point cloud, 'd' is 3, representing the coordinates of each point. When the input is point cloud annotated with touch code, 'd' is 19, which includes the coordinates plus the 16-bit touch code.