PANOS: Payload-Aware Navigation in Offroad Scenarios

TL;DR

PANOS is introduced, a weakly supervised approach that integrates proprioception and exteroception from onboard sensing to achieve a stable gait while walking by a legged robot over various terrains and provides evidence of its adaptability over varying payloads.

Abstract

Nature has evolved humans to walk on different terrains by developing a detailed understanding of their physical characteristics. Similarly, legged robots need to develop their capability to walk on complex terrains with a variety of task-dependent payloads to achieve their goals. However, conventional terrain adaptation methods are susceptible to failure with varying payloads. In this work, we introduce PANOS, a weakly supervised approach that integrates proprioception and exteroception from onboard sensing to achieve a stable gait while walking by a legged robot over various terrains. Our work also provides evidence of its adaptability over varying payloads. We evaluate our method on multiple terrains and payloads using a legged robot. PANOS improves the stability up to 44% without any payload and 53% with 15 lbs payload. We also notice a reduction in the vibration cost of 20% with the payload for various terrain types when compared to state-of-the-art methods.

Figures (5)

• Figure 1: Variance plots from PCA-based measure of proprioception on the same terrain with varying payloads. We can observe the uncertainty from these plots when characterizing the terrain. Therefore, PANOS unwraps the proprioceptive measures and establishes a one-to-one contextual relationship between the proprioception and terrain type (using exteroception).
• Figure 2: Overview of the Pipeline: PANOS inputs a stream of images and proprioception data $\mathbf{P}_t$ (joints, hips, and, feet slips) recorded in an unsupervised fashion. The framework encodes these readings into two backbones DINOv2 oquab2023dinov2 and a vanilla encoder for proprioception resulting in sets of random sequences $S_t$ with visual tokens $\mathbf{F}_t^{\text{visual}}$ and proprioceptive features $\mathbf{F}_t^{\text{proprio}}$ stacked together. As an intermediate step, a pointer network is defined to assign the weighted confidence $\text{Confidence}_t$ between these sets and select the dominating ones to train. Finally, we use the trained contextual relationship $\mathbf{c}_t$ as the input to a neural network that predicts the optimal velocity.
• Figure 3: Sequences S3 and S6 selected from the context vector $\mathbf{c}_t$ based on their high confidence values helps in adapting high-level representation of specific terrains (grass and concrete).
• Figure 4: Raw foot slips from the legged robot's proprioception data $\mathbf{P}_t \in \mathbb{R}^{d_P}$ used in PANOS to supervise the training module. We can infer the variability of the robot's foot slips from the body frame (0 as a reference) occurring on different terrains w and w/o payload.
• Figure 5: Stability Modeling: Setup of IMU configuration used to measure the stability of different types of terrain. As a modeling parameter, we define the reliability of the setup by measuring the mean jerk $\bar{J}$ on three different terrains East(concrete), Medium(grass), and Hard (Gravel). The graph shown above shows the $\bar{J}$ acting on the five IMUs used in the setup while driving on three different terrains with three distinct properties.