Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Grasping Force Estimation for Markerless Visuotactile Sensors

Julio Castaño-Amoros, Pablo Gil

TL;DR

This work has designed and implemented several approaches to estimate the normal grasping force using different types of markerless visuotactile representations obtained from VBTS, and shows that the RGB visuotactile representation is a better input option than the depth image or a combination of the two for estimating normal grasping forces.

Abstract

Tactile sensors have been used for force estimation in the past, especially Vision-Based Tactile Sensors (VBTS) have recently become a new trend due to their high spatial resolution and low cost. In this work, we have designed and implemented several approaches to estimate the normal grasping force using different types of markerless visuotactile representations obtained from VBTS. Our main goal is to determine the most appropriate visuotactile representation, based on a performance analysis during robotic grasping tasks. Our proposal has been tested on the dataset generated with our DIGIT sensors and another one obtained using GelSight Mini sensors from another state-of-the-art work. We have also tested the generalization capabilities of our best approach, called RGBmod. The results led to two main conclusions. First, the RGB visuotactile representation is a better input option than the depth image or a combination of the two for estimating normal grasping forces. Second, RGBmod achieved a good performance when tested on 10 unseen everyday objects in real-world scenarios, achieving an average relative error of 0.125 +- 0.153. Furthermore, we show that our proposal outperforms other works in the literature that use RGB and depth information for the same task.

Grasping Force Estimation for Markerless Visuotactile Sensors

TL;DR

This work has designed and implemented several approaches to estimate the normal grasping force using different types of markerless visuotactile representations obtained from VBTS, and shows that the RGB visuotactile representation is a better input option than the depth image or a combination of the two for estimating normal grasping forces.

Abstract

Tactile sensors have been used for force estimation in the past, especially Vision-Based Tactile Sensors (VBTS) have recently become a new trend due to their high spatial resolution and low cost. In this work, we have designed and implemented several approaches to estimate the normal grasping force using different types of markerless visuotactile representations obtained from VBTS. Our main goal is to determine the most appropriate visuotactile representation, based on a performance analysis during robotic grasping tasks. Our proposal has been tested on the dataset generated with our DIGIT sensors and another one obtained using GelSight Mini sensors from another state-of-the-art work. We have also tested the generalization capabilities of our best approach, called RGBmod. The results led to two main conclusions. First, the RGB visuotactile representation is a better input option than the depth image or a combination of the two for estimating normal grasping forces. Second, RGBmod achieved a good performance when tested on 10 unseen everyday objects in real-world scenarios, achieving an average relative error of 0.125 +- 0.153. Furthermore, we show that our proposal outperforms other works in the literature that use RGB and depth information for the same task.

Paper Structure

This paper contains 16 sections, 6 equations, 11 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Methods
  4. Visuotactile Representations
  5. CNN Architectures for Normal Force Regression
  6. Experimental Setup
  7. Robotic Setup with Tactile Sensors
  8. Data Acquisition and Dataset Generation
  9. Metric Evaluation
  10. Experimental Results and Analysis
  11. Evaluation of the performance of the proposed methods
  12. Limitations of depth data for force regression task
  13. Experimentation with everyday objects
  14. Experimentation with non-symmetrical objects
  15. Comparison with other works from literature
  16. ...and 1 more sections

Figures (11)

  • Figure 1: Depth estimation with noise filtering from the RGB image
  • Figure 2: (a,c,e,g,i,k) are RGB images. (b,d,f,h,j,l) are filtered depth images. The applied forces were: (a,b) 4.317N, (c,d) 3.485N, (e,f) 8.858N, (g,h) 6.535N, (i,j) 9.192N, and (k,l) 14.406N
  • Figure 3: Our RGBmod method estimates force values from a single RGB tactile image. The tactile features from the second, third, and last layer are concatenated to estimate the grasping force
  • Figure 4: The RGBmod+D method takes the RGB and the filtered depth tactile images as input, processes them separately using RGBmod and D, and concatenates the tactile features to estimate the force value
  • Figure 5: Example of our robotic setup during data collection, where the robot grasps one of the indenters in our dataset while the RGB, depth, and ground truth force data are recorded
  • ...and 6 more figures