Learned Slip-Detection-Severity Framework using Tactile Deformation Field Feedback for Robotic Manipulation

TL;DR

The paper tackles slip in robotic manipulation by treating slip as a continuum rather than a binary event and introduces a learned framework that simultaneously detects slip and estimates its severity using real-time GelSight Mini tactile data. It derives a rich set of vector-field based tactile features—including velocity, divergence, curl, and normalized contact area—and trains separate models for slip detection (Random Forest and Gradient Boosting) and slip severity estimation (LSTM and MLP), integrating these signals into a tactile feedback loop with a PD gripper controller. The approach demonstrates high slip-detection accuracy (near 99% in cross-validation) and strong severity-prediction performance (best MAE around 0.26 cm/s for unseen objects), enabling effective tactile-guided vertical sliding without external force control or prior object models. The results highlight the practical potential of tactile-only, learned slip handling for robust manipulation across diverse objects, while acknowledging hardware limitations and pointing to future work in adaptive force control and more capable grippers to extend the framework to angular slip and complex tasks.

Abstract

Safely handling objects and avoiding slippage are fundamental challenges in robotic manipulation, yet traditional techniques often oversimplify the issue by treating slippage as a binary occurrence. Our research presents a framework that both identifies slip incidents and measures their severity. We introduce a set of features based on detailed vector field analysis of tactile deformation data captured by the GelSight Mini sensor. Two distinct machine learning models use these features: one focuses on slip detection, and the other evaluates the slip's severity, which is the slipping velocity of the object against the sensor surface. Our slip detection model achieves an average accuracy of 92%, and the slip severity estimation model exhibits a mean absolute error (MAE) of 0.6 cm/s for unseen objects. To demonstrate the synergistic approach of this framework, we employ both the models in a tactile feedback-guided vertical sliding task. Leveraging the high accuracy of slip detection, we utilize it as the foundational and corrective model and integrate the slip severity estimation into the feedback control loop to address slips without overcompensating.

Figures (6)

• Figure 1: Summary of the Slip-Detection-Severity Framework: A robot executes an object handling task, during which tactile features from the GelSight Mini sensor are extracted in real time. These features simultaneously feed into the Slip Detection and Slip Severity models. Upon detecting slip, the feedback controller actively adjusts the gripper to mitigate slip severity.
• Figure 2: The figure depicts the deformation vector fields through the displacement of markers, highlighting the displacement vector components of a specific marker.
• Figure 3: Two tactile interaction scenarios, each with corresponding sensor frames and graphs (sensor mounted on a single finger), are presented: (a) Static grasping, where there is no relative motion between the object and the sensor, divergence increases noticeably. (b) Object rotation, introduces torsional stress, leading to simultaneous increases in both divergence and curl, as well as inhomogeneity in instantaneous area of contact, as observed in the graph.
• Figure 4: Schematic representation of the data collection pipeline for the Slip Detection experiment. The STATIC and GRASP scenarios detail the methodology for acquiring 'no-slip' data labeled 0, while the SLIP scenario illustrates the process for gathering data indicative of slip labeled 1. Both datasets are combined during the training phase for input into the slip detection model.
• Figure 5: Illustration of the data acquisition framework for the Slip Severity Estimation Model. The gripper, fitted with a GelSight Mini sensor, is programmed to slide over a fixed object. This setup synchronously captures tactile feedback and slip velocity data—the latter serving as ground truth—to train the neural network in assessing slip severity.