Step2Motion: Locomotion Reconstruction from Pressure Sensing Insoles

TL;DR

Step2Motion tackles the problem of reconstructing full-body locomotion using only insole sensor data, enabling unconstrained outdoor motion capture. The authors introduce a diffusion-based pose reconstruction framework conditioned on multimodal insole data, augmented by a body-part-aware cross-attention mechanism, and a separate IMU-driven displacement predictor for root motion. Through evaluations on UnderPressure and a new Step2Motion dataset, the method achieves accurate pose, low root-motion drift, and robust performance across diverse locomotion styles, including dancing. This approach advances accessible, robust motion capture for sports analysis, rehabilitation, and entertainment in real-world environments, by effectively leveraging foot-ground interactions captured in insoles.

Abstract

Human motion is fundamentally driven by continuous physical interaction with the environment. Whether walking, running, or simply standing, the forces exchanged between our feet and the ground provide crucial insights for understanding and reconstructing human movement. Recent advances in wearable insole devices offer a compelling solution for capturing these forces in diverse, real-world scenarios. Sensor insoles pose no constraint on the users' motion (unlike mocap suits) and are unaffected by line-of-sight limitations (in contrast to optical systems). These qualities make sensor insoles an ideal choice for robust, unconstrained motion capture, particularly in outdoor environments. Surprisingly, leveraging these devices with recent motion reconstruction methods remains largely unexplored. Aiming to fill this gap, we present Step2Motion, the first approach to reconstruct human locomotion from multi-modal insole sensors. Our method utilizes pressure and inertial data-accelerations and angular rates-captured by the insoles to reconstruct human motion. We evaluate the effectiveness of our approach across a range of experiments to show its versatility for diverse locomotion styles, from simple ones like walking or jogging up to moving sideways, on tiptoes, slightly crouching, or dancing.

Figures (8)

• Figure 1: Overview of the diffusion-based motion reconstruction process conditioned to insole sensor data. The method starts with some unit gaussian noise $\mathbf{m}_T$ and returns the denoised motion sequence $\mathbf{m}_0$ after $T$ iterations. The diffusion block is executed each iteration: given a noisy input motion sample $\mathbf{m}_t$ at timestep $t$, the output is the denoised pose $\mathbf{m}_{t-1}$ at timestep $t-1$. The input is divided into three representations, $\mathbf{m_L}$, $\mathbf{m_R}$, $\mathbf{m_B}$, to facilitate part-wise attention within the Transformer network. The insole data $\mathbf{c}$ is partitioned into eight components to be used as heads for multi-head cross-attention. Sinusoidal positional encodings transformers are employed to encode motion temporality and the current diffusion timestep. An additional Transformer network predicts the displacements $\mathbf{m_d}$ of the corresponding motion $\mathbf{m}$ from the IMU readings of both feet.
• Figure 2: Comparison on a jump followed by walking motion sequence. Root motion is aligned to the ground truth (with an offset for visualization) to highlight pose differences. Our full method (cyan) accurately captures the jump trajectory and overall motion, while the Transformer baseline (orange) exhibits significant jitter and the MLP baseline (yellow) produces overly smooth motion. Removing the insole multi-head cross-attention (red) leads to a degradation in accuracy. The bottom row provides close-up views.
• Figure 3: Reconstructing locomotion styles. Our method allows the reconstruction of different motion styles only using insole data, from walking and running to crouching, walking sideways, or moving on tiptoes.
• Figure 4: Qualitative comparison on various motion sequences. Root motion is aligned to the ground truth (with a slight offset for visualization purposes) to highlight pose quality differences. These animations were not used for training. The top image shows the same jump as Figure \ref{['fig:baseline:jump']} but captured at the highest point. The middle and bottom images show squatting and walking motion, respectively. Our full method (cyan) accurately reconstructs various motions, including in-place movements like squatting. Removing the Insole MHA (red) leads to errors, particularly in sequences where distinct sensor information is crucial, such as the squat (relying on pressure) and walking (relying on IMU). The MLP (yellow) and Transformer (orange) baselines exhibit limitations in capturing these motions and overall, smoothed-out results.
• Figure 5: Accumulated root positional error over time. The figure demonstrates the limitations of double integration, pressure-only inputs, MLP architectures, and integrating displacement prediction into the diffusion process. Our full method, with or without pressure data, consistently shows the lowest error.