Tactile Probabilistic Contact Dynamics Estimation of Unknown Objects

TL;DR

The paper addresses rapid tactile identification of contact dynamics for unknown objects in partially known environments by jointly estimating contact geometries and physical parameters using a DeepSDF geometry prior and a particle-filter Bayesian estimator within a quasi-static rigid-body simulator. An information-theoretic active exploration strategy guides data collection to maximize $EIG$ (based on $D_{KL}$), improving sample efficiency. The approach demonstrates sub-Newton force prediction errors in physical experiments and strong accuracy in simulation across unknown walls and friction, with fewer than 30 exploration moves. This framework enables online, robust manipulation of unknown objects using tactile sensing in open-world settings, and opens avenues for extension to 3D, improved sample efficiency, and RL-driven exploration for downstream tasks.

Abstract

We study the problem of rapidly identifying contact dynamics of unknown objects in partially known environments. The key innovation of our method is a novel formulation of the contact dynamics estimation problem as the joint estimation of contact geometries and physical parameters. We leverage DeepSDF, a compact and expressive neural-network-based geometry representation over a distribution of geometries, and adopt a particle filter to estimate both the geometries in contact and the physical parameters. In addition, we couple the estimator with an active exploration strategy that plans information-gathering moves to further expedite online estimation. Through simulation and physical experiments, we show that our method estimates accurate contact dynamics with fewer than 30 exploration moves for unknown objects touching partially known environments.

Figures (6)

• Figure 1: Our work considers estimating the contact dynamics of unknown planar rigid objects rigidly attached to a robot with an unknown rigid transform in a fully rigid partially known environment. Our method uses a particle filter with a compact representation of object geometry to quickly estimate the contact dynamics accurately based on tactile measurement only.
• Figure 2: Overview of the proposed estimation process. The robot manipulates an unknown object in a rigid environment with unknown surface height and wall position. Our method leverages DeepSDF park2019deepsdf (a compact learned SDF) as a representation of object geometry allowing joint estimation of geometry and physical parameters using a particle filter. We then employ an active exploration strategy to select actions which give the maximum expected information gain measured in terms of the Kullback–Leibler (KL) divergence.
• Figure 3: Experiment Setup for both simulation (left) and physical world (right). In simulation, the environment used is a flat surface with an unknown height and a wall with an unknown position (Wall). The objects tested are 2D slices of YCB objects ycb. In physical experiments, the Mustard_Bottle, Mug, and Lemon are tested in the Wall environment. Contact wrenches are measured by the internal end-effector force torque sensor (F/T) of the UR5e robot arm.
• Figure 4: Estimation progress for the Master_Chef_Can object for Active exploration and Expert exploration policies. The estimations of wrench, wall position, and floor position during the exploration trajectory are plotted with standard deviation. The trajectories and the current distribution of geometries are also visualized, with contact forces shown as green lines. The geometry distribution is the weighted indicator function ($+1$ outside of the object and $-1$ inside the object) for the top 100 particles in $\Theta$ with the largest weights.
• Figure 5: Evaluation of the dynamics model estimated by Active exploration, and Expert exploration policies for the Master_Chef_Can object. The estimations of wrenches are plotted with standard deviation. The evaluation trajectory is also visualized at the top.