Learning Humanoid Locomotion over Challenging Terrain

TL;DR

A learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrain and using a transformer model to predict the next action based on the history of proprioceptive observations and actions is presented.

Abstract

Humanoid robots can, in principle, use their legs to go almost anywhere. Developing controllers capable of traversing diverse terrains, however, remains a considerable challenge. Classical controllers are hard to generalize broadly while the learning-based methods have primarily focused on gentle terrains. Here, we present a learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrain. Our method uses a transformer model to predict the next action based on the history of proprioceptive observations and actions. The model is first pre-trained on a dataset of flat-ground trajectories with sequence modeling, and then fine-tuned on uneven terrain using reinforcement learning. We evaluate our model on a real humanoid robot across a variety of terrains, including rough, deformable, and sloped surfaces. The model demonstrates robust performance, in-context adaptation, and emergent terrain representations. In real-world case studies, our humanoid robot successfully traversed over 4 miles of hiking trails in Berkeley and climbed some of the steepest streets in San Francisco.

Figures (11)

• Figure 1: Humanoid locomotion over challenging terrain. Our controller can successfully traverse a range of outdoor terrain, including steep, rough, rutted, rocky, wet, muddy, and sand.
• Figure 2: Berkeley hikes. Our robot has successfully completed over 4 miles of real hiking trails in Berkeley. (A-D) We show basic statistics and examples from four different hikes.
• Figure 3: San Francisco streets. Our robot can successfully navigate some of the steepest streets in San Francisco, which are also among some of the steepest streets in the world.
• Figure 4: Terrain representations. We observe that the representations of our model cluster based on the terrain the robot is walking on. These representations emerge as a byproduct of learning to walk. We did not design, supervise, or impose any constraints on the representations.
• Figure 5: Kinematic adaptation. We show the joint positions of the key hip and knee joints when walking over terrain with different slope. We see that our controller adapts its gait based on the terrain slope it is walking over. This has not been pre-programmed during training.