Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VibWalk: Mapping Lower-limb Haptic Experiences of Everyday Walking

Shih Ying-Lei, Dongxu Tang, Weiming Hu, Sang Ho Yoon, Yitian Shao

TL;DR

VibWalk presents a foot-worn system that records wideband vibrations during natural walking to decode ground materials, grain size, and road conditions, achieving up to ~95% material identification accuracy within users and ~87% cross-user accuracy, complemented by GPS-based haptic maps. The approach leverages a dual-sensor setup (ACC and MIC) and a ResNet-based model to fuse temporal and spectral features, with careful data collection from 31 participants across 18 materials. Key findings include strong correlations between vibration spectra and grain size, robust performance under indoor conditions, and notable sensitivity to environmental noise, mitigated by hardware design (vibration plates) and modality fusion. The work demonstrates practical pathways for mapping pedestrian haptic experiences and providing data for urban monitoring, while outlining future work in edge computing, immersive feedback, wearability, and joint human-vehicle sensing.

Abstract

Walking is among the most common human activities where the feet can gather rich tactile information from the ground. The dynamic contact between the feet and the ground generates vibration signals that can be sensed by the foot skin. While existing research focuses on foot pressure sensing and lower-limb interactions, methods of decoding tactile information from foot vibrations remain underexplored. Here, we propose a foot-equipped wearable system capable of recording wideband vibration signals during walking activities. By enabling location-based recording, our system generates maps of haptic data that encode information on ground materials, lower-limb activities, and road conditions. Its efficacy was demonstrated through studies involving 31 users walking over 18 different ground textures, achieving an overall identification accuracy exceeding 95\% (cross-user accuracy of 87\%). Our system allows pedestrians to map haptic information through their daily walking activities, which has potential applications in creating digitalized walking experiences and monitoring road conditions.

VibWalk: Mapping Lower-limb Haptic Experiences of Everyday Walking

TL;DR

VibWalk presents a foot-worn system that records wideband vibrations during natural walking to decode ground materials, grain size, and road conditions, achieving up to ~95% material identification accuracy within users and ~87% cross-user accuracy, complemented by GPS-based haptic maps. The approach leverages a dual-sensor setup (ACC and MIC) and a ResNet-based model to fuse temporal and spectral features, with careful data collection from 31 participants across 18 materials. Key findings include strong correlations between vibration spectra and grain size, robust performance under indoor conditions, and notable sensitivity to environmental noise, mitigated by hardware design (vibration plates) and modality fusion. The work demonstrates practical pathways for mapping pedestrian haptic experiences and providing data for urban monitoring, while outlining future work in edge computing, immersive feedback, wearability, and joint human-vehicle sensing.

Abstract

Walking is among the most common human activities where the feet can gather rich tactile information from the ground. The dynamic contact between the feet and the ground generates vibration signals that can be sensed by the foot skin. While existing research focuses on foot pressure sensing and lower-limb interactions, methods of decoding tactile information from foot vibrations remain underexplored. Here, we propose a foot-equipped wearable system capable of recording wideband vibration signals during walking activities. By enabling location-based recording, our system generates maps of haptic data that encode information on ground materials, lower-limb activities, and road conditions. Its efficacy was demonstrated through studies involving 31 users walking over 18 different ground textures, achieving an overall identification accuracy exceeding 95\% (cross-user accuracy of 87\%). Our system allows pedestrians to map haptic information through their daily walking activities, which has potential applications in creating digitalized walking experiences and monitoring road conditions.

Paper Structure

This paper contains 46 sections, 4 equations, 11 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Footwear Sensing
  4. Vibration Sensing Techniques
  5. On-foot Vibration Sensing
  6. Sensing Principle
  7. Hardware Design
  8. Combining ACC and MIC for Wideband Vibration Sensing
  9. Noise Robustness Assessment
  10. Data-driven VibWalk Model for Decoding Ground Information
  11. Ground Materials
  12. Experiment Setup and Data Collection
  13. Ground surface materials
  14. Participants and apparatus
  15. Data collection procedure
  16. ...and 31 more sections

Figures (11)

  • Figure 1: The system overview of VibWalk. (a) An example data collection scene including two ground materials: wooden board and carpet. VibWalk synchronously record signals from the two accelerometers (ACC) and the microphone (MIC) attached to the shoe. (b) The system extracts the spectral features and the low-frequency feature of the windowed vibration data to decode walking information, such as ground materials and grain sizes.
  • Figure 2: Assessment of ACC and MIC sensing. (a) A vibration transmission plate is used to tap and slide on four different materials. The first purple line on each picture represents the start of the tapping, and the second purple line represents the start of the sliding. (b) Tapping and sliding the plate under two different noise conditions. 40 cm means a 60 dB noise source is positioned 40 cm from the sensors, and 80 cm means the noise is 80 cm away from the sensors.
  • Figure 3: The hardware design of VibWalk. (a) The fundamental sensing components of VibWalk; (b) VibWalk worn on a user; (c) Individual components, in which a1-a3 and b1-b4 are electronic components, c1-c2 are the mechanical components for affixing our accelerometers to shoes, and d1-d2 are mechanical components for housing the Raspberry Pi. a1-Raspberry Pi 5, a2-sensor contact plate, a3-radiator, a4-Raspberry Pi fixed shell, b1-microphone (MIC), b2-camera, b3-accelerometer (ACC), b4-sensor connection cable, c1-forefoot vibration transmitter plate, c2-rearfoot vibration transmitter plate, d1-square housing to cover the Raspberry Pi, d2-support component between a user’s lower leg and the hardware and carry the camera.
  • Figure 4: The photos of 18 ground materials used to study material classification. We divided the materials into 5 categories: Ornament, Grain, Floor, Paving, Stroma.
  • Figure 5: Our VibWalk model was designed based on a ResNet framework. We have a total of five BasicBlock, each with two convolutional layers. The numbers of channels in those blocks are 32, 64, 128, 256, and 512, respectively. In addition, for the MLP(Multi Layer Perceptron) layer, the first layer has an input channel of 3200 and an output channel of 128, and all subsequent layers have an input channel and an output channel of 128. BasicBlock comprises the following key components: Convolutional and Normalization Layers, Residual Connection (Shortcut), Auxiliary Input Processing (Optional), Forward Propagation
  • ...and 6 more figures