Diffusion-based Inverse Model of a Distributed Tactile Sensor for Object Pose Estimation

TL;DR

This work tackles vision-free object pose estimation in tactile-rich manipulation by learning a diffusion-based inverse tactile sensor model conditioned on distributed tactile observations. It couples a DDIM-based inverse model with SDF-based contact constraints and a geometry-aware projection to generate multimodal pose hypotheses, which are then integrated into a belief-preserving particle filter via belief-informed injection. Across simulated and real-world planar tasks, the approach achieves higher sample efficiency and accuracy than local sampling baselines, sustains multimodal beliefs, and demonstrates robustness to unmodeled contact dynamics. By combining data-efficient diffusion learning with explicit geometric reasoning and probabilistic filtering, it provides a practical framework for tactile-only perception in occluded or cluttered environments, with extensions to 6-DoF and multi-object scenarios as promising future directions.

Abstract

Tactile sensing provides a promising sensing modality for object pose estimation in manipulation settings where visual information is limited due to occlusion or environmental effects. However, efficiently leveraging tactile data for estimation remains a challenge due to partial observability, with single observations corresponding to multiple possible contact configurations. This limits conventional estimation approaches largely tailored to vision. We propose to address these challenges by learning an inverse tactile sensor model using denoising diffusion. The model is conditioned on tactile observations from a distributed tactile sensor and trained in simulation using a geometric sensor model based on signed distance fields. Contact constraints are enforced during inference through single-step projection using distance and gradient information from the signed distance field. For online pose estimation, we integrate the inverse model with a particle filter through a proposal scheme that combines generated hypotheses with particles from the prior belief. Our approach is validated in simulated and real-world planar pose estimation settings, without access to visual data or tight initial pose priors. We further evaluate robustness to unmodeled contact and sensor dynamics for pose tracking in a box-pushing scenario. Compared to local sampling baselines, the inverse sensor model improves sampling efficiency and estimation accuracy while preserving multimodal beliefs across objects with varying tactile discriminability.

Figures (12)

• Figure 1: Particle-based belief during a simulated pose estimation experiment. Starting from a uniform prior, a cylindrical end-effector equipped with a distributed tactile sensor makes sequential contacts with a static drill. The diffusion-based inverse sensor model generates tactile-conditioned pose hypotheses at each contact, as the belief converges to the true pose. The ground-truth object is shown in green, and the belief particles in blue, with opacity proportional to their weights.
• Figure 2: Overview of the proposed framework. Offline, a denoising diffusion implicit model (DDIM) is trained on tactile observations from a simulated contact dataset. Online, the trained model generates pose hypotheses conditioned on incoming tactile observations from a sensorized end-effector, with SDF-based projection enforcing contact constraints. Generated hypotheses are injected as proposal particles in a particle filter that updates a belief over object pose across sequential contacts.
• Figure 3: Left: cylindrical end-effector mounted on a Franka Emika 7-DoF manipulator and equipped with a distributed tactile sensor (CySkin). The sensor is covered by a conductive fabric layer, followed by a compliant elastomer layer and a fabric sleeve. Center: a single triangular skin module. Right: full distributed taxel array.
• Figure 4: Top: 2D slice of the signed distance field (SDF) of a drill object. The colormap shows signed distance values, with the zero-level set (object boundary) indicated by a black contour. Arrows in the left half-plane denote SDF gradients used for contact projection. Bottom: Expected taxel activation under the geometric sensor model, shown as a colormap for the region designated by a green box.
• Figure 5: Inference pipeline of the inverse sensor model. Left: ground-truth contact configuration and corresponding tactile observation. Middle: denoised DDIM particles. Right: final pose hypotheses after SDF-based constraint projection. Particles are shown in blue with opacity proportional to likelihood under the observation model. The ground-truth is shown in green.