Tactile Pose Estimation and Policy Learning for Unknown Object Manipulation

TL;DR

This work tackles tactile pose estimation and manipulation for unknown category-level objects by learning a tactile observation model $p(s_t|I_l,I_r)$ and embedding it in a discrete Bayes filter with a deterministic transition, followed by belief-based policy learning using PPO. The perception and control components are trained in simulation and transferred to a real UR5 with visuo-tactile sensors, achieving pose localization at $2\,\mathrm{mm}$ and $1^\circ$ resolution and an $81\%$ real-world success rate on a bottle-opening task. Results show that belief-based policies (TPN-PPO) are more sample-efficient and robust to unseen objects than LSTM-based baselines, highlighting the advantages of decoupled perception and control in partially observable, contact-rich manipulation. The approach demonstrates practical sim-to-real transfer with minimal domain randomization and offers a pathway for precise tactile localization to enhance vision-based manipulation in real-world robotics.

Abstract

Object pose estimation methods allow finding locations of objects in unstructured environments. This is a highly desired skill for autonomous robot manipulation as robots need to estimate the precise poses of the objects in order to manipulate them. In this paper, we investigate the problems of tactile pose estimation and manipulation for category-level objects. Our proposed method uses a Bayes filter with a learned tactile observation model and a deterministic motion model. Later, we train policies using deep reinforcement learning where the agents use the belief estimation from the Bayes filter. Our models are trained in simulation and transferred to the real world. We analyze the reliability and the performance of our framework through a series of simulated and real-world experiments and compare our method to the baseline work. Our results show that the learned tactile observation model can localize the pose of novel objects at 2-mm and 1-degree resolution for position and orientation, respectively. Furthermore, we experiment on a bottle opening task where the gripper needs to reach the desired grasp state.

Figures (5)

• Figure 1: Experimental Setup - Our method learns to estimate a robotic gripper pose with respect to manipulated objects. Later, we use this estimator to learn manipulation policies. The experimental setup consists of a UR5 arm, a parallel gripper, and two visuo-tacile sensors mounted on the gripper fingers. The robot grasps the object to perform the pose estimation, then, moves gripper to the desired pose to solve manipulation tasks.
• Figure 2: Top Image: Simulation environment - Bottom-left Image: Examples from simulated objects - Bottom-right Image: Real-world objects
• Figure 3: The System Framework -- Our proposed system has two main components: (a) a factored Bayes filter with a learned observation model and (b) a policy-value network that uses belief estimations to reach desired goals. The observation model uses a ResNet network to encode tactile images and produces the state-likelihood probabilities. The deterministic transition model predicts the belief at the next time step which is then multiplied and normalized with the likelihood probabilities. Finally, the policy network takes the belief along with the goal to output actions.
• Figure 4: The Policy Learning Curves -- This figure shows the success rates and mean episode lengths over the course of policy learning. We train the policies on a dataset with 50 bottles and evaluate during the training with 10 unseen bottles. The training is averaged over 4 different random seeds. The policies trained with belief estimation outperforms the policies with the recurrent network in terms of sample-efficiency and final performance.
• Figure 5: Tactile Image Augmentation - We augment the images in the simulation with the background image of the real sensor for improving the sim-to-real performance.