Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Novel Bioinspired Neuromorphic Vision-based Tactile Sensor for Fast Tactile Perception

Omar Faris, Mohammad I. Awad, Murana A. Awad, Yahya Zweiri, Kinda Khalaf

Abstract

Tactile sensing represents a crucial technique that can enhance the performance of robotic manipulators in various tasks. This work presents a novel bioinspired neuromorphic vision-based tactile sensor that uses an event-based camera to quickly capture and convey information about the interactions between robotic manipulators and their environment. The camera in the sensor observes the deformation of a flexible skin manufactured from a cheap and accessible 3D printed material, whereas a 3D printed rigid casing houses the components of the sensor together. The sensor is tested in a grasping stage classification task involving several objects using a data-driven learning-based approach. The results show that the proposed approach enables the sensor to detect pressing and slip incidents within a speed of 2 ms. The fast tactile perception properties of the proposed sensor makes it an ideal candidate for safe grasping of different objects in industries that involve high-speed pick-and-place operations.

A Novel Bioinspired Neuromorphic Vision-based Tactile Sensor for Fast Tactile Perception

Abstract

Tactile sensing represents a crucial technique that can enhance the performance of robotic manipulators in various tasks. This work presents a novel bioinspired neuromorphic vision-based tactile sensor that uses an event-based camera to quickly capture and convey information about the interactions between robotic manipulators and their environment. The camera in the sensor observes the deformation of a flexible skin manufactured from a cheap and accessible 3D printed material, whereas a 3D printed rigid casing houses the components of the sensor together. The sensor is tested in a grasping stage classification task involving several objects using a data-driven learning-based approach. The results show that the proposed approach enables the sensor to detect pressing and slip incidents within a speed of 2 ms. The fast tactile perception properties of the proposed sensor makes it an ideal candidate for safe grasping of different objects in industries that involve high-speed pick-and-place operations.
Paper Structure (18 sections, 5 equations, 11 figures)

This paper contains 18 sections, 5 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Hardware Methods
  3. Event-based Camera
  4. Markers Design
  5. System Assembly
  6. Fabrication Process
  7. Software Methods
  8. Events Pre-processing
  9. Events Aggregation
  10. Noise Reduction
  11. Final Heatmap
  12. Deep Learning Architecture
  13. Experiments
  14. Results
  15. Effect of Longer Events History
  16. ...and 3 more sections

Figures (11)

  • Figure 1: The proposed Neuromorphic Vision-Based Tactile Sensor (a) the realized system, (b) the events (sensor's data) (c) the proposed sensor integrated on a robotic parallel gripper
  • Figure 2: A 3D CAD render of a single bio-inspired marker in the finger: (a) Isometric view of the marker highlighting the marker protrusion. The protrusion design is inspired by the human finger ridges (b) Front view of the marker, and (c) Top view of the marker. The dimensions (in mm) are as follows: $h_1 = 4.80, d_1 = 2.10, w_1 = 40.00, w_2 = 12.00, w_3 = 4.00, w_4 = 7.20, t_1 = 0.50, t_2 = 0.80, l_1 = 4.00, l_2 = 4.20, l_3 = 2.40$.
  • Figure 3: Assembly of the proposed sensor (a) Exploded view of the sensor components and (b) Full assembly of the sensor body. (c) physical model of the sensor
  • Figure 4: Sample frames before (left) and after (right) applying the sigmoid function for noise reduction during (a) pressing and (b) slippage.
  • Figure 5: Flow of the proposed approach for processing the events and classifying the heatmap-based frames, along with the structure of the CNN used for the grasping stage classificaitons.
  • ...and 6 more figures