Robot Learning of 6 DoF Grasping using Model-based Adaptive Primitives

TL;DR

This work parametrizing the two remaining, lateral degrees of freedom of the primitives is applied to the task of 6 DoF bin picking, introducing a model-based controller to calculate angles that avoid collisions, maximize the grasp quality while keeping the uncertainty small.

Abstract

Robot learning is often simplified to planar manipulation due to its data consumption. Then, a common approach is to use a fully-convolutional neural network to estimate the reward of grasp primitives. In this work, we extend this approach by parametrizing the two remaining, lateral Degrees of Freedom (DoFs) of the primitives. We apply this principle to the task of 6 DoF bin picking: We introduce a model-based controller to calculate angles that avoid collisions, maximize the grasp quality while keeping the uncertainty small. As the controller is integrated into the training, our hybrid approach is able to learn about and exploit the model-based controller. After real-world training of 27000 grasp attempts, the robot is able to grasp known objects with a success rate of over 92% in dense clutter. Grasp inference takes less than 50ms. In further real-world experiments, we evaluate grasp rates in a range of scenarios including its ability to generalize to unknown objects. We show that the system is able to avoid collisions, enabling grasps that would not be possible without primitive adaption.

Figures (9)

• Figure 2: The robot avoids a possible collision with the bin by adapting the two lateral degrees of freedom by a model-based controller. The remaining planar grasping is learned in a self-supervised manner from real-world experiments. We evaluate our hybrid approach using the task of bin picking. Our system includes a robotic arm (1), a RGBD stereo camera (2), a two-finger gripper (3), and two bins with a multitude of objects (4).
• Figure 3: Our algorithm maps a point cloud via orthographic RGBD images (a) to a $\mathit{SE}(3)$ grasp point with an additional gripper stroke $d_m$ (b). The coordinate system uses an extrinsic rotation $a$ around $z$, and then intrinsic rotations $b$ and $c$ around $x^\prime$ and $y^\prime$ respectively.
• Figure 4: Sectional drawings of depth profiles $s(x, y)$ (blue) to illustrate the controller derivation for a given grasp point (red). In (c), the hatched area marks an undercut from the camera view from above.
• Figure 5: During inference, a FCNN estimates the reward for a grid of grasp poses. For a single rotation and single primitive, the result can be interpreted as a reward heatmap. A model-based controller calculates the lateral DoF $(z, b, c)$ based on the selected planar grasp $(x, y, a, m)$. During training, the reward of the adapted grasp is fed into the FCNN.
• Figure 6: Examples of successful grasps, showing the RGBD image window $s^\prime$. The projected approach vector (green) represents the angles $b$ and $c$, the result of the model-based adaption. Furthermore, the grasp and its stroke is shown (red).