TexSenseGAN: A User-Guided System for Optimizing Texture-Related Vibrotactile Feedback Using Generative Adversarial Network

TL;DR

TexSenseGAN addresses texture-related vibrotactile generation by integrating a SRResNet-based GAN with a human-in-the-loop Differential Subspace Search, enabling users to steer a high-dimensional latent space via a simple 1-D slider. The method leverages an auxiliary classifier and conditional information to produce class-controllable spectrograms while a DSS mapping preserves intuitive user control, with $z^* = \arg\max_{z\in Z} E(G(z))$ guiding iterative updates. Experiments on the LMT108-derived dataset show generated vibrations can be distinguished from targets and correlate with real textures, achieving 58% accuracy in a five-class task for generated samples and revealing a meaningful but imperfect alignment between subjective similarity and objective spectral similarity. Overall, TexSenseGAN demonstrates a viable, user-guided approach to texture-like vibrotactile feedback, offering a foundation for more intuitive, high-fidelity haptic rendering in VR and game interfaces, and pointing to future work with diffusion models and broader device integration.

Abstract

Vibration rendering is essential for creating realistic tactile experiences in human-virtual object interactions, such as in video game controllers and VR devices. By dynamically adjusting vibration parameters based on user actions, these systems can convey spatial features and contribute to texture representation. However, generating arbitrary vibrations to replicate real-world material textures is challenging due to the large parameter space. This study proposes a human-in-the-loop vibration generation model based on user preferences. To enable users to easily control the generation of vibration samples with large parameter spaces, we introduced an optimization model based on Differential Subspace Search (DSS) and Generative Adversarial Network (GAN). With DSS, users can employ a one-dimensional slider to easily modify the high-dimensional latent space to ensure that the GAN can generate desired vibrations. We trained the generative model using an open dataset of tactile vibration data and selected five types of vibrations as target samples for the generation experiment. Extensive user experiments were conducted using the generated and real samples. The results indicated that our system could generate distinguishable samples that matched the target characteristics. Moreover, we established a correlation between subjects' ability to distinguish real samples and their ability to distinguish generated samples.

Figures (15)

• Figure 1: The structure of our optimization system. Users can easily control the high-dimensional latent vector with a 1-D slider, and the generator can give spectrograms according to the latent vector. Users can compare the tactile sensation of the generated vibration and the real target, and update the optimizer by the slider position corresponding to the closest result to the target.
• Figure 2: The structure of the GAN model in this research. The model can generate vibrotactile spectrograms from the latent space built by the ResNet-50 encoder.
• Figure 3: The structure of the SRResNet-based GAN.
• Figure 4: The vibrotactile presenting device in this research. The vibrator was placed in a foam block as shown in (a), and a piece of acrylic board was attached to the top of the vibrator for users to touch, as shown in (b).
• Figure 5: The user interface of the optimization.