Learning Tactile Insertion in the Real World
Daniel Palenicek, Theo Gruner, Tim Schneider, Alina Böhm, Janis Lenz, Inga Pfenning, Eric Krämer, Jan Peters
TL;DR
Problem: enabling robust tactile-insertion control under partial observability. Approach: end-to-end Dreamer-v3 RL using GelSight tactile inputs on a Franka manipulator, with simulation via Taxim and a real autonomous-reset platform. Contributions: (i) a tactile insertion simulation setup based on Taxim, (ii) a real robot platform with autonomous resetting for unsupervised training, and (iii) a preliminary evaluation showing tactile feedback improves learning in both simulation and reality. Findings: tactile information enables higher success rates and faster learning, though real-world gains are somewhat smaller due to hardware softness and sim-to-real gaps. Impact: provides a scalable platform for evaluating tactile learning in dexterous manipulation and benchmarks for other RL algorithms.
Abstract
Humans have exceptional tactile sensing capabilities, which they can leverage to solve challenging, partially observable tasks that cannot be solved from visual observation alone. Research in tactile sensing attempts to unlock this new input modality for robots. Lately, these sensors have become cheaper and, thus, widely available. At the same time, the question of how to integrate them into control loops is still an active area of research, with central challenges being partial observability and the contact-rich nature of manipulation tasks. In this study, we propose to use Reinforcement Learning to learn an end-to-end policy, mapping directly from tactile sensor readings to actions. Specifically, we use Dreamer-v3 on a challenging, partially observable robotic insertion task with a Franka Research 3, both in simulation and on a real system. For the real setup, we built a robotic platform capable of resetting itself fully autonomously, allowing for extensive training runs without human supervision. Our preliminary results indicate that Dreamer is capable of utilizing tactile inputs to solve robotic manipulation tasks in simulation and reality. Furthermore, we find that providing the robot with tactile feedback generally improves task performance, though, in our setup, we do not yet include other sensing modalities. In the future, we plan to utilize our platform to evaluate a wide range of other Reinforcement Learning algorithms on tactile tasks.