PROBE: Proprioceptive Obstacle Detection and Estimation while Navigating in Clutter

TL;DR

This work tackles obstacle prediction under occlusion by enabling a legged robot to reconstruct a 2D scene using only proprioceptive history. It introduces PROBE, a Transformer-based Obstacle Reconstruction Module (ORM) that maps SE(2) pose and joint-state histories to obstacle parameters, including 2D pose, dimensions, and mobility, even for nested interactions behind occluders. Trained in simulation with randomized domains and validated on a real Unitree Go1, PROBE achieves measurable rotated IoU and pose/shape accuracy for movable and static obstacles, demonstrating that proprioception can substitute for vision in challenging SAR-like environments. The approach offers a practical pathway to reliable obstacle awareness when sensing modalities are limited, with future work aimed at broader shapes and multimodal sensor integration.

Abstract

In critical applications, including search-and-rescue in degraded environments, blockages can be prevalent and prevent the effective deployment of certain sensing modalities, particularly vision, due to occlusion and the constrained range of view of onboard camera sensors. To enable robots to tackle these challenges, we propose a new approach, Proprioceptive Obstacle Detection and Estimation while navigating in clutter PROBE, which instead relies only on the robot's proprioception to infer the presence or absence of occluded rectangular obstacles while predicting their dimensions and poses in SE(2). The proposed approach is a Transformer neural network that receives as input a history of applied torques and sensed whole-body movements of the robot and returns a parameterized representation of the obstacles in the environment. The effectiveness of PROBE is evaluated on simulated environments in Isaac Gym and with a real Unitree Go1 quadruped robot.

Figures (6)

• Figure 1: The setup considered in PROBE involves a Go1 robot dog and obstacles that can potentially obstruct its path. Some planar obstacles, such as the long frontal box in the image, are movable, while others are fixed to the ground. A transformer network reconstructs the locations and sizes of the obstacles, including the occluded ones, from a history of proprioceptive data that the robot receives while exploring the scene without vision.
• Figure 2: (Left) Environment setup in reality (top) and simulation (bottom), best viewed in color. The robot's workspace is bounded in a box of dimensions $w_\text{env} \times l_\text{env}$, and the goal region (purple) is defined as the set of all locations that satisfy $\{x > K \}$, i.e., all locations that are beyond $K$ meters in the direction of the robot's initial orientation. The yellow long obstacle is movable, while the red obstacle is static. The robot has to move the yellow box in front of it to reach the goal region. (Middle-Right) A hierarchical control policy is executed to navigate the robot so it may explore the environment's properties and successfully reach the goal region. Concurrently, the proposed obstacle reconstruction module (ORM) is a Transformer neural network that uses localization and proprioception history from the robot to predict object position and dimensions through interaction.
• Figure 3: Given as input a sequence of the robot's joint positions, velocities, applied torques, and poses (Top), the Obstacle Reconstruction Module (ORM, middle) is a Transformer-based neural network that outputs the sizes and poses for the different movable (ground truth visualized in yellow) and static (ground truth visualized in red) obstacles in the scene (Bottom, best viewed in color). The reconstruction for the movable and static obstacles are visualized in orange and blue, respectively, and their corresponding Rotated Intersection Over Union (See: Section \ref{['sec:experiments']}-C) values are reported. Higher values represent more accurate reconstructions.
• Figure 4: Examples of Easy (Left), Medium (Middle) and Hard (Right) simulated (top) and real-world (bottom) scenarios considered in the evaluation.
• Figure 5: An example execution of PROBE with a real Unitree Go1, best viewed in color. (Top) Snapshots of the experiment at different timestamps. (Bottom) Obstacle reconstruction returned by PROBE. As the experiment progresses, the robot (green) comes into direct contact with the movable obstacle (ground truth pose in yellow) and indirect contact with the static obstacles (ground truth poses in red). The predictions during the contact window for the movable and static obstacles are visualized in orange and blue, respectively.