VET: A Visual-Electronic Tactile System for Immersive Human-Machine Interaction

TL;DR

The paper tackles the challenge of achieving high-fidelity, bidirectional tactile interaction on a single interface by integrating electrical stimulation with vision-based tactile sensing. It introduces the VET system, a compact, screen-printed electrode film integrated onto a VBTS, enabling decoupled neural feedback and tactile sensing through a dual-channel stimulator and closed-loop control. The authors validate the approach with experiments on spatial electrotactile sensitivity and demonstrate immersive bidirectional interaction in Unity-based simulations and robotic teleoperation scenarios, highlighting improvements in immersion and control precision. This work enables more realistic and responsive haptic feedback for VR, teleoperation, and human–machine collaboration, laying the groundwork for scalable, closed-loop tactile interfaces.

Abstract

In the pursuit of deeper immersion in human-machine interaction, achieving higher-dimensional tactile input and output on a single interface has become a key research focus. This study introduces the Visual-Electronic Tactile (VET) System, which builds upon vision-based tactile sensors (VBTS) and integrates electrical stimulation feedback to enable bidirectional tactile communication. We propose and implement a system framework that seamlessly integrates an electrical stimulation film with VBTS using a screen-printing preparation process, eliminating interference from traditional methods. While VBTS captures multi-dimensional input through visuotactile signals, electrical stimulation feedback directly stimulates neural pathways, preventing interference with visuotactile information. The potential of the VET system is demonstrated through experiments on finger electrical stimulation sensitivity zones, as well as applications in interactive gaming and robotic arm teleoperation. This system paves the way for new advancements in bidirectional tactile interaction and its broader applications.

Figures (8)

• Figure 1: Illustration of the most critical components of the VET system. VET achieves both tactile sensing and haptic feedback on the same surface, which can be applied to multiple fields, such as virtual reality and teleoperation of robotic arms.
• Figure 2: Framework diagram of bidirectional tactile human-machine interaction. (a) Internal structure diagram of VBTS. (b) Components of VET and an introduction to their connection methods. (c) Physical demonstration diagram of the VET system.
• Figure 3: Schematic diagram of the electrical control system, which consists of three parts. The function generator generates the desired stimulation current AC1 and AC2, the current monitor ensures the consistency of output perception by feedback control, and the switch array manages the state of each electrode separately. ADC, analog-to-digital converter.
• Figure 4: The principles, methods, and effects of electrical stimulation. (a) Sensory corpuscles involved in skin electrical stimulation and their response frequency and spatial resolution. (b) Picture of the grounding electrode placement position. (c) When the grounding electrode is placed on the back, the perceived locations on the hand are under electrical stimulation currents of different polarities (positive and negative).
• Figure 5: Principle of electrode stimulation film. (a) Schematic diagram of finger pressing on VET. (b) Verification of film deformation capabilities. (c) Physical representation of the film after screen printing. (d) Comparison of internal visuotactile images of VET with and without electrical stimulation film.