Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring Object Rotation via Visuo-Tactile Segmentation

Julio Castaño, Pablo Gil

TL;DR

The paper tackles slip-induced rotation during robotic grasping by leveraging tactile sensing with DIGIT sensors. It introduces a two-stage pipeline: a PSPNet-based tactile segmentation network to isolate the contact region from RGB tactile images, followed by Skeleton Thinning-based geometry to estimate the rotation angle of the contact region, smoothed with a moving window. A public tactile-segmentation dataset with real DIGIT data (3675 images from 16 YCB objects) supports robust evaluation, showing Dice around 0.95–0.97 and IoU around 0.88–0.93, and a mean absolute rotational error of $1.85^ 0.96^ $ degrees on unseen objects, with a maximum seen error of $2.81^ $ degrees. The approach yields real-time performance and provides a practical route to preemptive grip adjustments to prevent object falls in manipulation tasks, while acknowledging limitations for circular or highly curved contact geometries and proposing future work to broaden applicability.

Abstract

When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotation caused by slippage. This can be prevented by obtaining tactile information that provides better knowledge on the physical properties of the grasping. In this paper, we estimate the rotation angle of a grasped object when slippage occurs. We implement a system made up of a neural network with which to segment the contact region and an algorithm with which to estimate the rotated angle of that region. This method is applied to DIGIT tactile sensors. Our system has additionally been trained and tested with our publicly available dataset which is, to the best of our knowledge, the first dataset related to tactile segmentation from non-synthetic images to appear in the literature, and with which we have attained results of 95% and 90% as regards Dice and IoU metrics in the worst scenario. Moreover, we have obtained a maximum error of 3 degrees when testing with objects not previously seen by our system in 45 different lifts. This, therefore, proved that our approach is able to detect the slippage movement, thus providing a possible reaction that will prevent the object from falling.

Measuring Object Rotation via Visuo-Tactile Segmentation

TL;DR

The paper tackles slip-induced rotation during robotic grasping by leveraging tactile sensing with DIGIT sensors. It introduces a two-stage pipeline: a PSPNet-based tactile segmentation network to isolate the contact region from RGB tactile images, followed by Skeleton Thinning-based geometry to estimate the rotation angle of the contact region, smoothed with a moving window. A public tactile-segmentation dataset with real DIGIT data (3675 images from 16 YCB objects) supports robust evaluation, showing Dice around 0.95–0.97 and IoU around 0.88–0.93, and a mean absolute rotational error of degrees on unseen objects, with a maximum seen error of degrees. The approach yields real-time performance and provides a practical route to preemptive grip adjustments to prevent object falls in manipulation tasks, while acknowledging limitations for circular or highly curved contact geometries and proposing future work to broaden applicability.

Abstract

When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotation caused by slippage. This can be prevented by obtaining tactile information that provides better knowledge on the physical properties of the grasping. In this paper, we estimate the rotation angle of a grasped object when slippage occurs. We implement a system made up of a neural network with which to segment the contact region and an algorithm with which to estimate the rotated angle of that region. This method is applied to DIGIT tactile sensors. Our system has additionally been trained and tested with our publicly available dataset which is, to the best of our knowledge, the first dataset related to tactile segmentation from non-synthetic images to appear in the literature, and with which we have attained results of 95% and 90% as regards Dice and IoU metrics in the worst scenario. Moreover, we have obtained a maximum error of 3 degrees when testing with objects not previously seen by our system in 45 different lifts. This, therefore, proved that our approach is able to detect the slippage movement, thus providing a possible reaction that will prevent the object from falling.
Paper Structure (13 sections, 3 equations, 9 figures, 4 tables)

This paper contains 13 sections, 3 equations, 9 figures, 4 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Estimating the Contact Region
  4. Slip Detection and Estimation of the Object Rotation
  5. METHODS
  6. Our Method for the Estimation of the Slippage Angle
  7. Variants of our Method for a Comparative and Ablation Study
  8. EXPERIMENTATION AND RESULTS
  9. Data Acquisition and Dataset Generation
  10. Performance Evaluation of our Proposal vs. other Approaches
  11. Objective and Subjective Comparison of our proposal vs State-of-the-Art Methods
  12. Discussion and Limitations of our Method
  13. CONCLUSIONS

Figures (9)

  • Figure 1: Scheme of our system combining both stages
  • Figure 2: Setup made up of an external camera, a UR5e robot arm with a 3F Robotiq gripper and DIGIT sensors mounted on the fingertips
  • Figure 3: Objects from YCB dataset used to generate our tactile segmentation dataset. https://github.com/AUROVA-LAB/aurova_grasping/tree/main/Tactile_sensing/Digit_sensor/Tactile_segmentation/dataset
  • Figure 4: Examples of tactile segmentation made by our TSNN. The first row corresponds to the raw tactile images, the second row is the ground truth segmented images and the third row corresponds to the predicted contact region
  • Figure 5: Description of rotation angle calculation during the manipulation task with object 2 from our set of experimentation
  • ...and 4 more figures