The Role of Tactile Sensing for Learning Reach and Grasp
Boya Zhang, Iris Andrussow, Andreas Zell, Georg Martius
TL;DR
This work investigates how different tactile sensing modalities affect reinforcement-learning–driven reach and 2‑finger antipodal grasping, using a parallelized simulation pipeline and real‑robot tests with a Minsight‑like tactile setup. By comparing sensing area and data types (e.g., binary, magnitude, and full force vectors) under perfect and degraded vision, the study finds that tactile information is most beneficial when vision is imperfect, but high spatial resolution is not strictly necessary; a sum‑force vector input often yields the strongest gains and supports generalization. Across sim and sim‑to‑real tests, tactile cues improve robustness and transfer, though a notable sim‑to‑real gap remains, partly due to contact dynamics and sensor fidelity. The results offer practical guidance for designing tactile sensing in 2‑finger RL grasping systems and motivate future work on blind grasping and multi‑finger manipulation in more diverse environments.
Abstract
Stable and robust robotic grasping is essential for current and future robot applications. In recent works, the use of large datasets and supervised learning has enhanced speed and precision in antipodal grasping. However, these methods struggle with perception and calibration errors due to large planning horizons. To obtain more robust and reactive grasping motions, leveraging reinforcement learning combined with tactile sensing is a promising direction. Yet, there is no systematic evaluation of how the complexity of force-based tactile sensing affects the learning behavior for grasping tasks. This paper compares various tactile and environmental setups using two model-free reinforcement learning approaches for antipodal grasping. Our findings suggest that under imperfect visual perception, various tactile features improve learning outcomes, while complex tactile inputs complicate training.