Sound of Touch: Active Acoustic Tactile Sensing via String Vibrations

Abstract

Distributed tactile sensing remains difficult to scale over large areas: dense sensor arrays increase wiring, cost, and fragility, while many alternatives provide limited coverage or miss fast interaction dynamics. We present Sound of Touch, an active acoustic tactile-sensing methodology that uses vibrating tensioned strings as sensing elements. The string is continuously excited electromagnetically, and a small number of pickups (contact microphones) observe spectral changes induced by contact. From short-duration audio signals, our system estimates contact location and normal force, and detects slip. To guide design and interpret the sensing mechanism, we derive a physics-based string-vibration simulator that predicts how contact position and force shift vibration modes. Experiments demonstrate millimeter-scale localization, reliable force estimation, and real-time slip detection. Our contributions are: (i) a lightweight, scalable string-based tactile sensing hardware concept for instrumenting extended robot surfaces; (ii) a physics-grounded simulation and analysis tool for contact-induced spectral shifts; and (iii) a real-time inference pipeline that maps vibration measurements to contact state.

Figures (10)

• Figure 1: Sound of Touch leverages vibration as tactile sensing. Contact with a continuously excited string produces characteristic spectral changes that encode contact location, force, and slip, enabling real-time tactile inference from short-duration audio signals.
• Figure 2: (A) A contact at position $x$ with force $F$ alters the string’s effective vibrating lengths and tension, inducing characteristic shifts in modal frequencies. (B) Dual EBow-like electromagnetic drivers provide continuous excitation, while two pickup microphones capture complementary vibration responses that enable separation of contact location and force.
• Figure 3: Resonant-frequency modulation by contact. (A, D) Theoretical trends showing asymmetric frequency shifts for position and monotonic shifts for force. (B, E) Simulated spectra reproducing these modal shifts. (C, F) Real-world measurements exhibiting complex harmonic structures and overtones.
• Figure 4: Learning architecture. Short audio windows are transformed into spectral features and encoded by a frozen Audio Spectrogram Transformer (AST). The resulting embeddings are shared by lightweight task-specific heads for contact location, normal force, contact detection, and slip detection.
• Figure 5: Hardware design of the string-based tactile sensor. The system integrates an aluminum frame supporting a steel guitar string, dual electromagnetic EBows for continuous excitation, and contact microphones for high-frequency vibration acquisition.