Pinching Visuo-haptic Display: Investigating Cross-Modal Effects of Visual Textures on Electrostatic Cloth Tactile Sensations

TL;DR

This work addresses how visual texture cues influence tactile perception during fabric interaction in VR by coupling a physical electrostatic cloth tactile display with a synchronized virtual cloth (Pinching Visuo-haptic Display). The method modulates tactile feedback through voltage $0$–$300$ V and frequency- or texture-driven cues, using four real fabrics rendered with four VR textures in a 18-participant study. Results indicate that visual textures modulate stiffness and warmth perceptions—voile increases stiffness while toweling shifts warmth—alongside a trending effect of denim on roughness, while thickness remains robust against visual modulation. This demonstrates that visual overlays can expand the expressive range of tactile fabric displays, informing VR shopping and virtual fitting experiences with richer cross-modal cues.

Abstract

This paper investigates how visual texture presentation influences tactile perception when interacting with electrostatic cloth displays. We propose a visuo-haptic system that allows users to pinch and rub virtual fabrics while feeling realistic frictional sensations modulated by electrostatic actuation. Through a user study, we examined the cross-modal effects between visual roughness and perceived tactile friction. The results demonstrate that visually rough textures amplify the perceived frictional force, even under identical electrostatic stimuli. These findings contribute to the understanding of multimodal texture perception and provide design insights for haptic feedback in virtual material interfaces.

Figures (7)

• Figure 1: Example of an online shopper interacts tangibly with the fabric of the virtual clothes in the VR with the help of ECC in the real world.
• Figure 2: Top-The type of interaction gesture and its respective direction of motion shown in green arrow: (a)pinching (b)stroking (c)scrunching. Bottom-The corresponding gesture in VR.
• Figure 3: Schematic diagram of the pinching visuo-haptic display. The red rectangle represents the VR modality, while the blue rectangle represents the real world. In VR, the fabric visuals appear at the same position as the ECC in the real world. The ECC is driven by a voltage driver module regulated by an Arduino. A real fabric panel is also prepared for comparisons.
• Figure 4: Electrostatic tactile modulation in PTD. Voltage applied to conductive cloth generates electrostatic forces that induce periodic skin deformation through varying frictional forces. Tactile perception varies as parameters are adjusted: (a) zero voltage produces negligible friction, while (b) low and (c) high voltages correspond to proportional increases in perceived frictional force. Frequency modulation independently affects texture perception - (d) low frequencies (50 Hz) produce a sense of roughness, while (e) high frequencies (200 Hz) produce a sense of finer texture.
• Figure 5: Left: Our proposed visuo-haptic setup that involves the PTD driver, a VR HMD and a ECC which is being hung using a hanger with a spatial tracker attached to it. Right: A closeup shot of the PTD driver consisting of an Arduino, Tactile Driver, and power supply.