Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuning

TL;DR

This work tackles the challenge of generalizing agile legged locomotion across unstructured terrains by integrating nine expert RL skills into a single foundation policy through multi-expert distillation and later refining it with RL. Depth images replace elevation maps, aided by a robust depth-noise model to bridge sim-to-real transfer, and the policy is repeatedly extended with new terrains via fine-tuning. Empirical results show that distillation alone underperforms on complex terrains, but RL fine-tuning recovers and often surpasses expert performance, with active-perception adaptations emerging as the robot learns to improve obstacle visibility. Real-world deployment on the ANYmal D demonstrates robust navigation in indoor and outdoor settings, validating the approach’s practicality and highlighting opportunities for further improvements in perception, memory, and scalability to even more terrains.

Abstract

Legged robots are well-suited for navigating terrains inaccessible to wheeled robots, making them ideal for applications in search and rescue or space exploration. However, current control methods often struggle to generalize across diverse, unstructured environments. This paper introduces a novel framework for agile locomotion of legged robots by combining multi-expert distillation with reinforcement learning (RL) fine-tuning to achieve robust generalization. Initially, terrain-specific expert policies are trained to develop specialized locomotion skills. These policies are then distilled into a unified foundation policy via the DAgger algorithm. The distilled policy is subsequently fine-tuned using RL on a broader terrain set, including real-world 3D scans. The framework allows further adaptation to new terrains through repeated fine-tuning. The proposed policy leverages depth images as exteroceptive inputs, enabling robust navigation across diverse, unstructured terrains. Experimental results demonstrate significant performance improvements over existing methods in synthesizing multi-terrain skills into a single controller. Deployment on the ANYmal D robot validates the policy's ability to navigate complex environments with agility and robustness, setting a new benchmark for legged robot locomotion.

Figures (9)

• Figure 1: Deployment of our policy on the ANYmal D robot in real-world environments. The robot performs a variety of motions to cross the different unstructured obstacles. A single policy is used in all deployments.
• Figure 2: Description of our approach. The training is decomposed into three stages. First, 9 individual skills are trained separately using RL. The skills use elevation maps as perceptive input. Second, the skills are distilled into a single network using supervised learning and the Dagger approach. The distilled policy receives depth images instead of elevation maps. We use the 9 skills and terrains of the first phase during distillation. Finally, the distilled policy is fine-tuned using RL. During this phase, there is no supervision from experts, and the policy is trained on a combination of the 9 terrains of the previous phase as well as 15 new terrains, which are obtained from 3D scans of real-world piles of rubble in search-and-rescue training facilities. Only 4 out of 15 scans are shown in the figure.
• Figure 3: Architecture of the policy used during distillation and fine-tuning. Three types of inputs are given to the policy: 4 depth images, proprioceptive information, and task commands. The depth images are processed individually by a CNN with 3 convolutional and max-pool layers, followed by two fully connected layers with an output dimension of 64. The features of the last fully connected layer of all images are then concatenated with the vector of proprioceptive information and fed through 2 LSTM layers. Finally, the output of the LSTM is concatenated with the vectors of proprioception and task commands and fed through an MLP composed of 3 fully connected layers with ELU activation.
• Figure 4: Processing applied to simulated and real depth images. Simulated images are degraded using the following steps: 1) pixels surrounding edges are shuffled and/or removed, 2) random holes are added using slowly evolving Perlin noise, and 3) the image is blurred using a Gaussian filter. Real images are 4) clipped, downsampled, and cropped, and 5) blurred using the same Gaussian filter as in simulation.
• Figure 5: Reconstruction loss during distillation for two observation modalities (elevation map and depth images) and two network architectures with and without memory (LSTM and MLP, respectively). Average and standard deviation over 8 runs.