Synthesizing Grasps and Regrasps for Complex Manipulation Tasks

TL;DR

This work formalizes grasp and regrasp synthesis for complex manipulation tasks by representing task motions as a sequence of constant screw motions in $SE(3)$ and using partial point-clouds to identify task-dependent grasping regions. An algorithm computes the minimum number of grasps by transforming object point clouds along the motion plan, deriving grasping regions, and scoring overlaps with a threshold $\gamma_{th}$, yielding whether a single grasp suffices or regrasping is required. The approach is validated on real RGB-D data and robot experiments, achieving a practical success rate and demonstrating the potential to integrate motion constraints into grasp planning. It lays groundwork for robust, task-aware manipulation beyond traditional pick-and-place by coupling geometry, friction constraints, and screw-motion planning, with future work to account for manipulator joint limits and collision constraints.

Abstract

In complex manipulation tasks, e.g., manipulation by pivoting, the motion of the object being manipulated has to satisfy path constraints that can change during the motion. Therefore, a single grasp may not be sufficient for the entire path, and the object may need to be regrasped. Additionally, geometric data for objects from a sensor are usually available in the form of point clouds. The problem of computing grasps and regrasps from point-cloud representation of objects for complex manipulation tasks is a key problem in endowing robots with manipulation capabilities beyond pick-and-place. In this paper, we formalize the problem of grasping/regrasping for complex manipulation tasks with objects represented by (partial) point clouds and present an algorithm to solve it. We represent a complex manipulation task as a sequence of constant screw motions. Using a manipulation plan skeleton as a sequence of constant screw motions, we use a grasp metric to find graspable regions on the object for every constant screw segment. The overlap of the graspable regions for contiguous screws are then used to determine when and how many times the object needs to be regrasped. We present experimental results on point cloud data collected from RGB-D sensors to illustrate our approach.

Figures (4)

• Figure 1: Example task considered in this paper where the robot has to pivot a CheezIt box three times. The partial point cloud of the object is extracted using an eye-in-hand configuration (top left). The sequence of pivoting motions is specified by the motion plan $\mathcal{G}$ (top right). Our approach enables the robot to compute the minimum number of grasps required to execute specified motion indicating that there is a need to regrasp the object after the initial two pivoting motions (bottom).
• Figure 2: Solution overview with pivoting a CheezIt box three times. Using the complete point cloud description of the CheezIt box available at the initial pose $\mathcal{O}_{{\bf g}_1}$ and the motion plan $\mathcal{G}$ we compute the set of point cloud representations $\mathcal{O}_{\mathcal{G}}$ and the set of ideal grasping regions $\mathcal{C}$, visualized in yellow (a-d). By sequentially comparing the computed grasping regions, in the appropriate reference frame, we see that we need a minimum of $2$ grasps (e-g); Grasp 1 (red) for the first two pivots and Grasp 2 (green) for the last pivot (h-i).
• Figure 3: Result where two grasps, i.e., one regrasping suffice to perform the task given by the plan skeleton $\{slide, pivot, slide\}$ using the partial point cloud of a Ritz cracker box. One grasp (green) for the first sliding motion and one (red) for subsequent pivoting and sliding.
• Figure 4: Results where a single grasp is sufficient to perform the task (a) Plan skeleton $\{slide, pivot, pickup\}$ using the partial point cloud of a CheezIt box. (b) Pouring using the partial point cloud of the Pringles container. The first two images show the acquisition of a kinesthetic demonstration and extracted task constraints as a sequence of constant screw motions.