Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TiltXter: CNN-based Electro-tactile Rendering of Tilt Angle for Telemanipulation of Pasteur Pipettes

Miguel Altamirano Cabrera, Jonathan Tirado, Aleksey Fedoseev, Oleg Sautenkov, Vladimir Poliakov, Pavel Kopanev, Dzmitry Tsetserukou

TL;DR

TiltXter tackles the challenge of perceiving and manipulating deformable objects in remote teleoperation by using a CNN to classify pipette tilt from tactile data and by rendering tilt cues through electro-tactile patterns. The system combines high-density tactile sensing with a two-stage rendering pipeline: downsizing sensor data and applying CNN-informed tactile patterns, delivered via electro-stimulation. Experimental results show strong tilt recognition with CNN (88%+ accuracy) and a dramatic rise in teleoperation success (from about 53% to over 92%) when CNN-based rendering is used, with favorable NASA-TLX metrics. This approach advances safe, dexterous remote manipulation of deformable instruments and points to scalable future datasets and remote-lab applications.

Abstract

The shape of deformable objects can change drastically during grasping by robotic grippers, causing an ambiguous perception of their alignment and hence resulting in errors in robot positioning and telemanipulation. Rendering clear tactile patterns is fundamental to increasing users' precision and dexterity through tactile haptic feedback during telemanipulation. Therefore, different methods have to be studied to decode the sensors' data into haptic stimuli. This work presents a telemanipulation system for plastic pipettes that consists of a Force Dimension Omega.7 haptic interface endowed with two electro-stimulation arrays and two tactile sensor arrays embedded in the 2-finger Robotiq gripper. We propose a novel approach based on convolutional neural networks (CNN) to detect the tilt of deformable objects. The CNN generates a tactile pattern based on recognized tilt data to render further electro-tactile stimuli provided to the user during the telemanipulation. The study has shown that using the CNN algorithm, tilt recognition by users increased from 23.13\% with the downsized data to 57.9%, and the success rate during teleoperation increased from 53.12% using the downsized data to 92.18% using the tactile patterns generated by the CNN.

TiltXter: CNN-based Electro-tactile Rendering of Tilt Angle for Telemanipulation of Pasteur Pipettes

TL;DR

TiltXter tackles the challenge of perceiving and manipulating deformable objects in remote teleoperation by using a CNN to classify pipette tilt from tactile data and by rendering tilt cues through electro-tactile patterns. The system combines high-density tactile sensing with a two-stage rendering pipeline: downsizing sensor data and applying CNN-informed tactile patterns, delivered via electro-stimulation. Experimental results show strong tilt recognition with CNN (88%+ accuracy) and a dramatic rise in teleoperation success (from about 53% to over 92%) when CNN-based rendering is used, with favorable NASA-TLX metrics. This approach advances safe, dexterous remote manipulation of deformable instruments and points to scalable future datasets and remote-lab applications.

Abstract

The shape of deformable objects can change drastically during grasping by robotic grippers, causing an ambiguous perception of their alignment and hence resulting in errors in robot positioning and telemanipulation. Rendering clear tactile patterns is fundamental to increasing users' precision and dexterity through tactile haptic feedback during telemanipulation. Therefore, different methods have to be studied to decode the sensors' data into haptic stimuli. This work presents a telemanipulation system for plastic pipettes that consists of a Force Dimension Omega.7 haptic interface endowed with two electro-stimulation arrays and two tactile sensor arrays embedded in the 2-finger Robotiq gripper. We propose a novel approach based on convolutional neural networks (CNN) to detect the tilt of deformable objects. The CNN generates a tactile pattern based on recognized tilt data to render further electro-tactile stimuli provided to the user during the telemanipulation. The study has shown that using the CNN algorithm, tilt recognition by users increased from 23.13\% with the downsized data to 57.9%, and the success rate during teleoperation increased from 53.12% using the downsized data to 92.18% using the tactile patterns generated by the CNN.
Paper Structure (14 sections, 8 figures, 4 tables)

This paper contains 14 sections, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. TiltXter System Overview
  3. High-Fidelity Tactile Sensory and Haptic Interface
  4. Electro-tactile Rendering
  5. CNN Architecture Description
  6. Model training and validation
  7. Experimental Evaluation of Tilt Perception
  8. Experiment on Human Perception of Pipette Tilt Angle
  9. Experiment on Electro-tactile Perception
  10. Downsized Direct Data Rendering Results
  11. Tactile Pattern Data Rendering Results
  12. Experiment on Teleoperated Grasping
  13. Experimental Result
  14. Conclusions and Future Work

Figures (8)

  • Figure 1: Tilt angle recognition by the TiltXter system with high-density tactile sensor array in the remote site.
  • Figure 2: The overall architecture of the telemanipulation system. Green arrows define a hardware integration; blue arrows define a control signal; yellow arrows define a feedback loop.
  • Figure 3: Set of tactile patterns represented on the electrode array corresponding to the index finger distribution.
  • Figure 4: CNN model architecture for tilt angle classification from the tactile sensor data.
  • Figure 5: Data samples from the tactile sensors on the left and right fingers of the gripper during its contact with the tilted pipette. The data from the left sensor was flipped horizontally for dataset consistency.
  • ...and 3 more figures