Rapid Locomotion via Reinforcement Learning

TL;DR

The paper tackles fast, robust legged locomotion on natural terrain by training an end-to-end velocity-conditioned policy in simulation and transferring it to the real MIT Mini Cheetah using a teacher-student framework with online system identification and automatic curriculum strategies. It introduces adaptive Box and Grid curricula to shape feasible high-speed tasks and uses a 5-layer network to map sensory history and velocity commands to joint positions, with a teacher encoder for dynamics and a student adapter for online inference. Real-world results show a 3.9 m/s indoor sprint, 3.4 m/s outdoor grass dash, and stable ice spinning, with ablations confirming the importance of online adaptation and curriculum design. The work demonstrates that reinforcement learning can achieve record agility on a small quadruped with minimal sensing, offering a scalable alternative to hand-engineered model-based controllers.

Abstract

Agile maneuvers such as sprinting and high-speed turning in the wild are challenging for legged robots. We present an end-to-end learned controller that achieves record agility for the MIT Mini Cheetah, sustaining speeds up to 3.9 m/s. This system runs and turns fast on natural terrains like grass, ice, and gravel and responds robustly to disturbances. Our controller is a neural network trained in simulation via reinforcement learning and transferred to the real world. The two key components are (i) an adaptive curriculum on velocity commands and (ii) an online system identification strategy for sim-to-real transfer leveraged from prior work. Videos of the robot's behaviors are available at: https://agility.csail.mit.edu/

Figures (5)

• Figure 1: An end-to-end learned controller enables the MIT Mini Cheetah to execute: (a) fast sprinting at 3.9m/s (top); (b) a rough terrain $10$-meter sprint at 3.4m/s; (c) high-speed spinning indoors; and (d) robust spinning on an icy patch. All behaviors are realized by a single neural network that is trained in simulation and deployed zero-shot in the real world.
• Figure 2: Our controller is a learned mapping from sensory inputs to desired joint positions. We parameterize it as 5-layer neural network $\pi_{\theta}$ with parameters $\theta$ optimized in simulation.
• Figure 3: (a) Forward and angular velocity tracking performance. The Grid Adaptive curriculum tracks a larger range of velocities than the Box Adaptive curriculum for all error thresholds. (b) Velocity tracking error in the forward axis (top) and yaw axis (bottom); darker is better. In each heatmap, the x-axis varies the forward velocity command between $[-6, 6m/s]$ and the y-axis varies the yaw rate between $[-6, 6rad/s]$. From left to right: No Curriculum fails to learn meaningful velocity control; its heatmaps correspond to a robot jittering in place, as its tracking error is equal to the command. Box Adaptive curriculum learns to control the robot but excludes extremes of the command space. Grid Adaptive curriculum covers a larger command area by accounting for the combined impact of running and turning speed on task difficulty.
• Figure 4: Online system identification reduces tracking error, particularly at high speeds. The command area increases as the error threshold is relaxed for teacher, student, and domain-randomized policies.
• Figure 5: (a) Increasing the magnitude of terrain roughness during training shrinks the range of commands the robot can successfully track -- the command area -- on flat ground. This reflects a trade-off between robustness to rough terrains and top speed on flat ground. (b) Histogram comparing the command area at error threshold $\epsilon = 0.3$, which corresponds to the gray vertical line on the left.