Precision-Focused Reinforcement Learning Model for Robotic Object Pushing

TL;DR

This paper improves the state-of-the-art by introducing a new memory-based vision-proprioception reinforcement learning model to push objects more precisely to target positions using fewer corrective movements.

Abstract

Non-prehensile manipulation, such as pushing objects to a desired target position, is an important skill for robots to assist humans in everyday situations. However, the task is challenging due to the large variety of objects with different and sometimes unknown physical properties, such as shape, size, mass, and friction. This can lead to the object overshooting its target position, requiring fast corrective movements of the robot around the object, especially in cases where objects need to be precisely pushed. In this paper, we improve the state-of-the-art by introducing a new memory-based vision-proprioception RL model to push objects more precisely to target positions using fewer corrective movements.

Figures (7)

• Figure 1: Schematic Overview. We adapt a state-of-the-art RL model to push objects more precisely to target positions by improving the sampling of object parameters and adding a gated recurrent unit to provide the agent with a memory.
• Figure 2: Model Architecture. We concatenate the Cartesian $(x,y)$ EE position with the latent object and goal states generated by an encoder trained prior to the RL agent. The observations of the entire episode are stored in a memory buffer to be processed by a GRU-layer. The hidden state of the most recent time step is used as the feature vector for actor and critic MLPs.
• Figure 3: Densities Sliding Friction Force. Mass and sliding friction coefficient sampled independently from uniform distributions vs. modified exponential distribution
• Figure 4: Visualization of the types of corrective movements
• Figure 5: Evaluation Results in Simulation. The results are obtained using a deterministic policy. All object parameters are sampled uniformly at random from the parameter ranges shown in Table \ref{['table_object_params']}, except the mass (range: $[0.001,0.01]$ kg) and sliding friction coefficient (range: $[0.2,0.3]$). Additionally, we had to adjust the minimum object height for all shapes to $0.052\,$m to avoid distorting the results by disappearing objects due to an unstable simulation.