Learning to Detect Slip through Tactile Estimation of the Contact Force Field and its Entropy

TL;DR

This work tackles slip detection during object manipulation by fusing tactile sensing with entropy-based features derived from GelSight images. By tracking marker displacements on the GelSight surface, the authors compute an entropy measure $E$ of the displacement distribution and its rate $\frac{\delta E}{\delta t}$, forming robust, object-agnostic slip indicators. Across 10 objects and 14{,}000 samples, entropy-based features improve classification accuracy (up to $95.61\%$ on average) and generalize better to unseen objects than velocity-based features, with real-time inference demonstrated on standard classifiers. A practical demonstration shows slip prevention during a book retrieval task, highlighting the approach's potential for improving robotic manipulation without heavy object priors.

Abstract

Detection of slip during object grasping and manipulation plays a vital role in object handling. Existing solutions primarily rely on visual information to devise a strategy for grasping. However, for robotic systems to attain a level of proficiency comparable to humans, especially in consistently handling and manipulating unfamiliar objects, integrating artificial tactile sensing is increasingly essential. We introduce a novel physics-informed, data-driven approach to detect slip continuously in real time. We employ the GelSight Mini, an optical tactile sensor, attached to custom-designed grippers to gather tactile data. Our work leverages the inhomogeneity of tactile sensor readings during slip events to develop distinctive features and formulates slip detection as a classification problem. To evaluate our approach, we test multiple data-driven models on 10 common objects under different loading conditions, textures, and materials. Our results show that the best classification algorithm achieves a high average accuracy of 95.61%. We further illustrate the practical application of our research in dynamic robotic manipulation tasks, where our real-time slip detection and prevention algorithm is implemented.

Figures (9)

• Figure 1: Experimental setup: An UR5e robot arm and a Robotiq parallel gripper, replacing the original fingertips with custom metallic adapters fitted with two GelSight tactile sensors. An Intel RealSense depth camera D435i is mounted atop the gripper.
• Figure 2: Design of adapter for housing Gelsight tactile sensors. A 45-degree flange extension was designed for the end-effector to extend the opening distance of the gripper to 85mm. The adapters are mounted on the Robotiq Hand-e adaptive gripper through four M2.5 screws.
• Figure 3: (a) Gripper grasping a T-handle hex key, (b) the displacement of individual markers overlaid on the tactile image, and (c) zoomed-in section of GelSight image, illustrates the gel deformation through arrows resulting from contact, denoted as $Dx_i$ and $Dy_i$, and referred to from hereon as the displacement field of the markers
• Figure 4: This histogram illustrates the distribution of marker flow, depicting the frequency of various marker lengths. Each bar in the histogram represents a distinct state, with its height indicating the relative occurrence of that state within the overall distribution.
• Figure 5: A slip trial was conducted to illustrate the change of entropy from the no contact to object, through the initial grasp, to the incipient slip, and ultimately to the loss of contact to object. Initially, a rectangular cardboard box was held between the grippers (embedded with GelSight sensor) of the robot, at this stage the entropy was almost zero. The grippers were brought closer together till there was an initial contact and a gentle grasp of the box, leading to a notable increase and subsequent stabilization of entropy. Following this, the robotic arm was maneuvered in a manner that induced slippage of the object. At the moment of slip, a sharp spike in entropy was observed and then the entropy returned zero once the robotic gripper with the tactile sensor lost complete contact.