Learning to Singulate Objects in Packed Environments using a Dexterous Hand

TL;DR

This paper proposes a novel method that involves a displacement-based state representation and a multi-phase reinforcement learning procedure that enables singulation using the 16-DOF Allegro Hand, outperforming alternative learning and non-learning methods.

Abstract

Robotic object singulation, where a robot must isolate, grasp, and retrieve a target object in a cluttered environment, is a fundamental challenge in robotic manipulation. This task is difficult due to occlusions and how other objects act as obstacles for manipulation. A robot must also reason about the effect of object-object interactions as it tries to singulate the target. Prior work has explored object singulation in scenarios where there is enough free space to perform relatively long pushes to separate objects, in contrast to when space is tight and objects have little separation from each other. In this paper, we propose the Singulating Objects in Packed Environments (SOPE) framework. We propose a novel method that involves a displacement-based state representation and a multi-phase reinforcement learning procedure that enables singulation using the 16-DOF Allegro Hand. We demonstrate extensive experiments in Isaac Gym simulation, showing the ability of our system to singulate a target object in clutter. We directly transfer the policy trained in simulation to the real world. Over 250 physical robot manipulation trials, our method obtains success rates of 79.2%, outperforming alternative learning and non-learning methods.

Figures (7)

• Figure 1: Overview of our proposed SOPE framework for object singulation, shown here with the Allegro Hand. The task involves three phases: isolating, grasping, and retrieving. We train a policy in simulation using phase-dependent reward functions. The policy uses the hand joints, the block positions, and the robot arm in the state representation. We directly transfer the trained policy to the real world.
• Figure 2: Visualizations of the "Block Information" part of the state representation $\mathbf{s}_t$. We show a clean view without the hand (left) followed by four examples of the state in physical experiments. The representation focuses on the target (colored red) and the displacement to its neighbors. By "block corners," we refer to the top of the AprilTag markers, or virtual points above the box endpoints, as indicated with the arrows.
• Figure 3: Comparison of our simulation (left) and real world (right) setups. In Isaac Gym simulation makoviychuk2021isaac, we set up a robot with an Allegro Hand and create a box with blocks in it. In the real world, we attach a physical Allegro Hand to a Franka, and set up similar blocks within a cardboard box. We attach AprilTag markers to the blocks.
• Figure 4: Policy performance curves for our method and alternatives; see Section \ref{['ssec:sim_exp_methods']} for more details. Each curve shows the average online success rate over the last 100 episodes. Results are averaged over 5 seeds, and shaded areas show standard deviations.
• Figure 5: Two examples of frame-by-frame visualizations of the non-learning baseline S2SS&P (target block colored red). Left: S2SS&P can succeed assuming sufficient space to push blocks. Right: S2SS&P can struggle in more constrained setups. Pushing the two adjacent blocks results in limited changes compared to the original setup and the fingers would "jam" into the blocks during the attempted grasp.