Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction

TL;DR

The paper tackles the difficulty of robust humanoid locomotion on complex terrains by integrating exteroceptive terrain perception with gait timing and full-body control. It introduces a perceptive framework that uses a downward-looking depth camera to produce a dense under-base height map via a lightweight U-Net, feeding a unified policy that outputs both joint commands and a gait-frequency action. A single-stage Successive Teacher–Student (S-TS) training regime enables efficient knowledge transfer from privileged to partial observations, yielding a robust end-to-end policy. The approach is validated on a full-sized 31-DoF humanoid (Oli) in simulation and real-world tests, demonstrating omnidirectional walking, stair and gap traversal, and zero-shot generalization to unseen terrains. This work advances terrain-aware, gait-adaptive humanoid locomotion with practical sim-to-real transfer capabilities.

Abstract

For full-size humanoid robots, even with recent advances in reinforcement learning-based control, achieving reliable locomotion on complex terrains, such as long staircases, remains challenging. In such settings, limited perception, ambiguous terrain cues, and insufficient adaptation of gait timing can cause even a single misplaced or mistimed step to result in rapid loss of balance. We introduce a perceptive locomotion framework that merges terrain sensing, gait regulation, and whole-body control into a single reinforcement learning policy. A downward-facing depth camera mounted under the base observes the support region around the feet, and a compact U-Net reconstructs a dense egocentric height map from each frame in real time, operating at the same frequency as the control loop. The perceptual height map, together with proprioceptive observations, is processed by a unified policy that produces joint commands and a global stepping-phase signal, allowing gait timing and whole-body posture to be adapted jointly to the commanded motion and local terrain geometry. We further adopt a single-stage successive teacher-student training scheme for efficient policy learning and knowledge transfer. Experiments conducted on a 31-DoF, 1.65 m humanoid robot demonstrate robust locomotion in both simulation and real-world settings, including forward and backward stair ascent and descent, as well as crossing a 46 cm gap. Project Page:https://ga-phl.github.io/

Figures (10)

• Figure 1: Full-sized humanoid robot Oli performing gait-adaptive locomotion on complex terrains: (a) climbing up and down long outdoor staircases; (b) going down stairs backwards; (c) crossing a 46 cm gap; and (d) climbing up stairs sideways
• Figure 2: Overview of the proposed Successive Teacher–Student (S-TS) framework and deployment pipeline. A teacher–student switch gate gradually transfers rollouts from the privileged teacher to the student. The unified policy outputs both joint actions and a scalar gait-frequency action. A downward-looking depth image is converted into an under-base height map by the perception module, which runs at 50 Hz together with the control policy.
• Figure 3: U-Net-based single-frame heightmap reconstruction network. The depth image is converted to a noisy base-centric heightmap and processed by a U-Net with two heads: A height head supervised by L1 loss and an edge head (training only) using BCE and Dice losses.
• Figure 4: This figure illustrates the pipeline of single-frame height map reconstruction using a U-Net model in deployment.
• Figure 5: Robot hardware "Limx Oli" in real-world (left) and simulation (right) setups, illustrating the robot's physical dimensions and degrees of freedom.