Tactile Estimation of Extrinsic Contact Patch for Stable Placement

TL;DR

This work tackles stable object placement under partial support by learning to interpret extrinsic contact from tactile sensing. It introduces a four-component pipeline: estimating a probabilistic extrinsic contact patch from co-located tactile and force data, assessing stability via a convex-hull test on high-probability contact regions, aggregating information across multiple interactions through a Bayesian-like update, and selecting actions to drive the object toward a more stable configuration. Empirical results on Bandu pieces show that fusing tactile and force signals improves contact-patch estimation, and that information aggregation yields high stability accuracy (around $90\%$) and meaningful stacking success, even with unknown environments. These findings enable robust, tactile-driven stacking of highly irregular objects in uncertain settings, with future work extending to more diverse geometries and relaxing prior geometric knowledge.

Abstract

Precise perception of contact interactions is essential for fine-grained manipulation skills for robots. In this paper, we present the design of feedback skills for robots that must learn to stack complex-shaped objects on top of each other (see Fig.1). To design such a system, a robot should be able to reason about the stability of placement from very gentle contact interactions. Our results demonstrate that it is possible to infer the stability of object placement based on tactile readings during contact formation between the object and its environment. In particular, we estimate the contact patch between a grasped object and its environment using force and tactile observations to estimate the stability of the object during a contact formation. The contact patch could be used to estimate the stability of the object upon release of the grasp. The proposed method is demonstrated in various pairs of objects that are used in a very popular board game.

Figures (7)

• Figure 1: Estimating extrinsic contact from tactile sensing: This work studies how extrinsic contact (indirect contact between a manipulated object and an environment) can be estimated in the context of stable placement of an object in the environment with partial support. The figure above shows an object stacking scenario using two lightweight wooden game pieces (from the popular Bandu puzzle). (Left) The contact area between the two objects being stacked is critical to the success of the stack. Vision-based tactile sensors mounted on the end effector and force-torque sensor provide us with a composite signal that includes both intrinsic (direct) and (partially observable) extrinsic contacts. Our key innovation is to propose a learning-based method to estimate the extrinsic contact patch using only the composite tactile signal and the knowledge of the force applied by the end effector. (Right) This enables the robot to stack a highly irregularly shaped object on top of a very unstable tower.
• Figure 2: Pipeline: Our method comprises four components. First, a robot probes the environment to establish contact between the grasped object and the target object upon which it must be stacked. During this probing phase, we acquire a sequence of force/torque measurements and tactile images. We then estimate the extrinsic contact patch and, in turn, the potential stability of the resultant configuration. Subsequently, we aggregate the information from multiple interactions to update the belief map of the contact state. We pick the action that maximizes the contact patch between the objects.
• Figure 3: Definition of the probabilistic contact patch. (Left) The displacement $(x, y)$ is added from the origin of the bottom object $O$ during data collection. This displacement and known contact surfaces of the two objects give the ground-truth contact surface $S$. (Right) The discretized contact patch $\hat{S}$ consists of a set of probabilities $p(s_j)$ that represents whether a specific position $s_j$ of the contact surface of the grasped object is in contact or not.
• Figure 4: The 3D printed board and Bandu pieces used in our experiments. (a) We use the 3D printed board for training data collection. The board includes small and large circles with diameters of $15$ and $25$ mm and one square whose length is $15$ mm. (b) The first two pieces on the left serve as the bottom objects (or the environment), while the subsequent three on the right are designated as the grasped (top) objects. These pieces have been assigned the following names: Short, Long, Mushroom, Barrel, and Pot from left to right.
• Figure 5: Distribution of contact patches: (a) Training data distribution with Pot as the grasped object and three different 3D printed shapes as the bottom objects (see Fig. \ref{['fig:bandu_piece']}). Each row shows the data obtained from different primitive shapes and each column shows the distribution of different data types: tactile displacements on the $XY$ axes (only shows the maximum absolute values from all 63 tracking markers), moments on the $XY$ axes and force $F_z$. The horizontal and vertical axes show the displacements randomly added during data collection (see Fig. \ref{['fig:definition']}), and the black circle or rectangle in each graph shows the contour of the bottom object. (b) Example contact patch sampled from the star points ($\bigstar$) in the left distributions. Although these contact patches are very different, the tactile signals look quite similar as seen in the data around the star point, showing the difficulty of the task; i.e., similar tactile signals can lead to very different contact patches.