Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual-modal Tactile E-skin: Enabling Bidirectional Human-Robot Interaction via Integrated Tactile Perception and Feedback

Shilong Mu, Runze Zhao, Zenan Lin, Yan Huang, Shoujie Li, Chenchang Li, Xiao-Ping Zhang, Wenbo Ding

TL;DR

The paper tackles the lack of integrated tactile perception and feedback in human–robot interaction by introducing a dual-modal e-skin that fuses magnetic tactile sensing with programmable haptic feedback. Leveraging a layered, flexible design with Hall-sensor arrays and vibrotactile actuators, it enables real-time, bidirectional tactile communication via Bluetooth between a human and a robot gripper. Key findings include high object-classification accuracy (~98.8%) from tactile data and a substantial improvement in fine-grain weighing resolution (~0.025 g) through programmable vibrations, demonstrated in grasping, weighing, and collaborative tasks. The work demonstrates the potential for immersive, precise, and scalable human–robot collaboration in complex manipulation tasks.

Abstract

To foster an immersive and natural human-robot interaction, the implementation of tactile perception and feedback becomes imperative, effectively bridging the conventional sensory gap. In this paper, we propose a dual-modal electronic skin (e-skin) that integrates magnetic tactile sensing and vibration feedback for enhanced human-robot interaction. The dual-modal tactile e-skin offers multi-functional tactile sensing and programmable haptic feedback, underpinned by a layered structure comprised of flexible magnetic films, soft silicone, a Hall sensor and actuator array, and a microcontroller unit. The e-skin captures the magnetic field changes caused by subtle deformations through Hall sensors, employing deep learning for accurate tactile perception. Simultaneously, the actuator array generates mechanical vibrations to facilitate haptic feedback, delivering diverse mechanical stimuli. Notably, the dual-modal e-skin is capable of transmitting tactile information bidirectionally, enabling object recognition and fine-weighing operations. This bidirectional tactile interaction framework will enhance the immersion and efficiency of interactions between humans and robots.

Dual-modal Tactile E-skin: Enabling Bidirectional Human-Robot Interaction via Integrated Tactile Perception and Feedback

TL;DR

The paper tackles the lack of integrated tactile perception and feedback in human–robot interaction by introducing a dual-modal e-skin that fuses magnetic tactile sensing with programmable haptic feedback. Leveraging a layered, flexible design with Hall-sensor arrays and vibrotactile actuators, it enables real-time, bidirectional tactile communication via Bluetooth between a human and a robot gripper. Key findings include high object-classification accuracy (~98.8%) from tactile data and a substantial improvement in fine-grain weighing resolution (~0.025 g) through programmable vibrations, demonstrated in grasping, weighing, and collaborative tasks. The work demonstrates the potential for immersive, precise, and scalable human–robot collaboration in complex manipulation tasks.

Abstract

To foster an immersive and natural human-robot interaction, the implementation of tactile perception and feedback becomes imperative, effectively bridging the conventional sensory gap. In this paper, we propose a dual-modal electronic skin (e-skin) that integrates magnetic tactile sensing and vibration feedback for enhanced human-robot interaction. The dual-modal tactile e-skin offers multi-functional tactile sensing and programmable haptic feedback, underpinned by a layered structure comprised of flexible magnetic films, soft silicone, a Hall sensor and actuator array, and a microcontroller unit. The e-skin captures the magnetic field changes caused by subtle deformations through Hall sensors, employing deep learning for accurate tactile perception. Simultaneously, the actuator array generates mechanical vibrations to facilitate haptic feedback, delivering diverse mechanical stimuli. Notably, the dual-modal e-skin is capable of transmitting tactile information bidirectionally, enabling object recognition and fine-weighing operations. This bidirectional tactile interaction framework will enhance the immersion and efficiency of interactions between humans and robots.
Paper Structure (11 sections, 1 equation, 8 figures, 1 table)

This paper contains 11 sections, 1 equation, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods and Design
  3. System framework overview
  4. Working principle
  5. Dual-modal e-skin fabrication
  6. Device cost
  7. Experiment
  8. Grasping recognition
  9. Programmable weighing operation
  10. Duplex human-robot immersion operation
  11. Discussion and Future Work

Figures (8)

  • Figure 1: (a) Schematic diagram of dual-modal e-skin bidirectional tactile interaction. (b) Picture of the fexible printed circuit board (FPCB) with functions of tactile sensing and actuation and Bluetooth module. (c) Photo of dual-modal e-skin.
  • Figure 2: Framework diagram of bidirectional tactile human-robot interaction. The dual-mode e-skin is placed on the human arm and the gripper of the robot respectively, and transmits tactile sensing and vibration feedback information bidirectionally and wirelessly.
  • Figure 3: (a-b) Working principle of the proposed dual-modal e-skin. (c) The impact of motor vibration on a single three-axis Hall sensor.
  • Figure 4: Fabrication process of the dual-modal e-skin.
  • Figure 5: Dual-modal e-skin-enabled grasping recognition. (a) Schematic diagram of a robot gripper assembling an e-skin. (b) Twelve types of common objects. (c) Array tactile information processing flow chart.
  • ...and 3 more figures