Proprioceptive Image: An Image Representation of Proprioceptive Data from Quadruped Robots for Contact Estimation Learning

TL;DR

This work introduces Proprioceptive Image (PI), a morphology-aware, 2D image representation of quadruped proprioceptive time-series that encodes temporal dynamics and robot structure to improve contact estimation. PI combines four sub-images—Slope Dynamics, Spikes Patterns, Polar Distance, and Global-Aware Local Shift—fed into a ConcatCNN that fuses multiple signals to predict foot–ground contact states with higher accuracy. Evaluations on real-world Lin2011LeggedRS data and MuJoCo simulations show the PI approach outperforms state-of-the-art baselines (e.g., MI-HGNN) while using a much shorter temporal window, indicating stronger generalization and robustness. The results suggest cross-modal encoding of proprioception into image space enhances learning performance and opens avenues for integration with graph-based models and reinforcement learning, as well as broader application to other robotic platforms.

Abstract

This paper presents a novel approach for representing proprioceptive time-series data from quadruped robots as structured two-dimensional images, enabling the use of convolutional neural networks for learning locomotion-related tasks. The proposed method encodes temporal dynamics from multiple proprioceptive signals, such as joint positions, IMU readings, and foot velocities, while preserving the robot's morphological structure in the spatial arrangement of the image. This transformation captures inter-signal correlations and gait-dependent patterns, providing a richer feature space than direct time-series processing. We apply this concept in the problem of contact estimation, a key capability for stable and adaptive locomotion on diverse terrains. Experimental evaluations on both real-world datasets and simulated environments show that our image-based representation consistently enhances prediction accuracy and generalization over conventional sequence-based models, underscoring the potential of cross-modal encoding strategies for robotic state learning. Our method achieves superior performance on the contact dataset, improving contact state accuracy from 87.7% to 94.5% over the recently proposed MI-HGNN method, using a 15 times shorter window size.

Figures (9)

• Figure 1: Examples of proprioceptive images over three time steps of a quadruped robot walking on uneven terrain. Each image encodes angular velocity, joint position, and foot position, illustrating how the representation varies across environments and supports learning of tasks such as foot contact estimation.
• Figure 2: Pipeline for constructing a Proprioceptive Image. Proprioceptive signal streams are grouped in triplets (X, Y, Z for trunk signals; HAA, HFE, KFE for leg signals) to form three-channel RGB inputs. Each windowed signal is processed by a temporal feature encoder that generates image-based encodings of slope dynamics, spike patterns, and GAF. In parallel, the local range of each signal triplet is used to create the final PI sub-image, named Global-Aware Local Shift Image, which is spatially concatenated with the PI Temporal Feature Encoder into a square image. Leg PIs are further arranged in a $LFRFLHRH$ layout to represent all four legs, while trunk PIs are used individually. Yellow circles labeled 'C' indicate channel concatenation, and circles labeled 'S' denote spatial concatenation.
• Figure 3: Examples of Slope Dynamics images from simulated joint position signal of a quadruped robot trotting.
• Figure 4: Examples of Spikes pattern images from simulated joint position of a quadruped robot trotting.
• Figure 5: Example of GAF images from simulated joint position signal of a quadruped robot trotting.