Humanoid Parkour Learning

TL;DR

This work tackles the challenge of humanoid parkour by presenting an end-to-end vision-based policy that operates entirely onboard and learns multiple parkour skills without motion priors. It combines a three-stage training pipeline—plane-walking pretraining, a diverse oracle parkour policy trained on scandots terrain, and distillation to a deployable student policy—with fractal-noise terrain and multi-GPU distillation to enable robust sim-to-real transfer on a real robot. Key findings show that onboard vision is essential, fractal-noise terrain improves generalization, and multi-GPU distillation significantly enhances learning efficiency and stability, while upper-limb overrides do not derail locomotion. The approach enables a Unitree H1 to perform tasks such as jumping onto platforms, leaping gaps, and running across varied terrains, marking a step toward practical, versatile humanoid parkour with reduced reliance on motion priors.

Abstract

Parkour is a grand challenge for legged locomotion, even for quadruped robots, requiring active perception and various maneuvers to overcome multiple challenging obstacles. Existing methods for humanoid locomotion either optimize a trajectory for a single parkour track or train a reinforcement learning policy only to walk with a significant amount of motion references. In this work, we propose a framework for learning an end-to-end vision-based whole-body-control parkour policy for humanoid robots that overcomes multiple parkour skills without any motion prior. Using the parkour policy, the humanoid robot can jump on a 0.42m platform, leap over hurdles, 0.8m gaps, and much more. It can also run at 1.8m/s in the wild and walk robustly on different terrains. We test our policy in indoor and outdoor environments to demonstrate that it can autonomously select parkour skills while following the rotation command of the joystick. We override the arm actions and show that this framework can easily transfer to humanoid mobile manipulation tasks. Videos can be found at https://humanoid4parkour.github.io

Figures (8)

• Figure 1: We present a single vision-based end-to-end whole-body-control parkour policy for humanoid robots that can jump on 0.42m platforms, leap over hurdles, 0.8m gaps, and overcome various terrains.
• Figure 2: We design 10 different types of terrain with controllable difficulty. By training on all these terrains, the oracle policy is able to react to almost all human-capable terrain.
• Figure 3: All the points are bound to each rigid body of the robot. The green points represent that part of the body does not penetrate the virtual obstacle. The red points represent that the body penetrates the virtual obstacle.
• Figure 4: From left to right is the process of simulating noise in simulation. From right to left is the process of pre-processing the depth image in the real world.
• Figure 5: Real-world quantitative results. Our parkour policy achieves the best performance in the 4 difficult tasks compared with the blind walking policy and the built-in MPC controller in Unitree H1. We use the user interface in H1's controller to raise its foot height so that it can overcome some of the obstacles. We run 10 trails for each testing configuration.