Self-Supervised Learning for Joint Pushing and Grasping Policies in Highly Cluttered Environments

TL;DR

A dual RL model is introduced, which presents high resilience in handling complicated scenes, reaching an average of 98% task completion in simulation and real-world scenes, outperforming recent state-of-the-art methods.

Abstract

Robots often face situations where grasping a goal object is desirable but not feasible due to other present objects preventing the grasp action. We present a deep Reinforcement Learning approach to learn grasping and pushing policies for manipulating a goal object in highly cluttered environments to address this problem. In particular, a dual Reinforcement Learning model approach is proposed, which presents high resilience in handling complicated scenes, reaching an average of 98% task completion using primitive objects in a simulation environment. To evaluate the performance of the proposed approach, we performed two extensive sets of experiments in packed objects and a pile of object scenarios with a total of 1000 test runs in simulation. Experimental results showed that the proposed method worked very well in both scenarios and outperformed the recent state-of-the-art approaches. Demo video, trained models, and source code for the results reproducibility purpose are publicly available. https://sites.google.com/view/pushandgrasp/home

Figures (8)

• Figure 1: Achieving Grasping Amidst Highly Cluttered Objects: The green target object, initially ungraspable due to surrounding blocks, is repositioned using pre-manipulation pushes by the robot. This strategic maneuver ensures a feasible grasp, showcasing the efficacy of our method.
• Figure 2: System Overview: Within the Gazebo environment, an RGB-D camera and dual-arm UR5e robot are integrated. The camera transforms sensory data into orthographic projections, generating goal and scene masks. Using 360° rotation, the grasp and push nets assess heightmaps and the goal mask, outputting Q values to dictate push or grasp actions.
• Figure 3: Post-mask Pixel $Q$-values: (left) Xu et al. (2021) flaws xu2021efficient; (right) our refined method.
• Figure 4: Grasp success rate versus the number of grasp epochs. The top two are related to the grasping network $\phi_g$ training, and the middle one is for the push network $\phi_p$ training. The last two are when the training is alternated between the grasp net and push net.
• Figure 5: The first column shows the original image; the next depicts the output angle and position; the last highlights the initial action. Case $1-9$: Simulation experiments with 9 distinct packed scenes featuring dense adversarial clutter; each scene's target is the green object. Case $10$: Typical scenes showcasing random $10$, $15$, and $20$ objects from left to right.