Grasp Multiple Objects with One Hand

TL;DR

This letter introduces MultiGrasp, a novel two-stage approach for multi-object grasping using a dexterous multi-fingered robotic hand on a tabletop, highlighting adaptability to new object configurations and tolerance for imprecise grasps.

Abstract

The intricate kinematics of the human hand enable simultaneous grasping and manipulation of multiple objects, essential for tasks such as object transfer and in-hand manipulation. Despite its significance, the domain of robotic multi-object grasping is relatively unexplored and presents notable challenges in kinematics, dynamics, and object configurations. This paper introduces MultiGrasp, a novel two-stage approach for multi-object grasping using a dexterous multi-fingered robotic hand on a tabletop. The process consists of (i) generating pre-grasp proposals and (ii) executing the grasp and lifting the objects. Our experimental focus is primarily on dual-object grasping, achieving a success rate of 44.13%, highlighting adaptability to new object configurations and tolerance for imprecise grasps. Additionally, the framework demonstrates the potential for grasping more than two objects at the cost of inference speed.

Figures (6)

• Figure 1: Overview of MultiGrasp. The pre-grasp pose proposal module (A, B) generates an optimal hand pose to grasp the target objects. A motion planning module (C) then plans the reaching trajectory from a flat hand to the desired pose. For lifting, a suite of specialist rl policies (D) are deployed, tailored for different object configurations. These policies are subsequently distilled (E) to develop a vision-based policy suitable for real-world application.
• Figure 2: Synthetic grasps using the augtmented dfc. From left to right: one (cols. 1-2), two (cols. 3-4), and three objects (col. 5).
• Figure 3: Generate grasps with desired palm direction. Given the desired palm direction (red arrow), the objects are rotated around the z-axis to align with the desired palm direction (black arrow). The pre-grasp pose is generated on the adjusted objects and is rotated to recover the pre-grasp under the initial object placement.
• Figure 4: Our framework supports grasping varying amounts (3-5) of objects. Each row demonstrates the object placement and the execution process for different numbers of objects.
• Figure 5: Experiment with a Shadow Hand. The sequence shows the phases of reaching, grasping, and lifting during the execution process.