MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion

TL;DR

The paper tackles robust quadrupedal payload transport across diverse terrains with unknown payloads by proposing an Adaptive RL framework that combines a nominal locomotion policy with an adaptive corrective policy. Trained in two phases, the approach uses a CE Net encoder to capture observation history and augments observations with foot-force estimates, enabling payload-aware adjustments without explicit parameter estimation. Phase-1 optimizes the nominal policy via PPO, while Phase-2 jointly trains both policies under dynamic payloads with a GRF-tracking reward, demonstrated through large-scale Isaac Gym simulations and real-world Unitree Go1 hardware trials on flat, slope, and stair terrains. The results show consistent improvements in body height and velocity command tracking, enhanced stability under payload changes, and a practical, gait-design-free method for real-world deployment in logistics and related fields.

Abstract

Quadrupedal robots are increasingly deployed for load-carrying tasks across diverse terrains. While Model Predictive Control (MPC)-based methods can account for payload variations, they often depend on predefined gait schedules or trajectory generators, limiting their adaptability in unstructured environments. To address these limitations, we propose an Adaptive Reinforcement Learning (RL) framework that enables quadrupedal robots to dynamically adapt to both varying payloads and diverse terrains. The framework consists of a nominal policy responsible for baseline locomotion and an adaptive policy that learns corrective actions to preserve stability and improve command tracking under payload variations. We validate the proposed approach through large-scale simulation experiments in Isaac Gym and real-world hardware deployment on a Unitree Go1 quadruped. The controller was tested on flat ground, slopes, and stairs under both static and dynamic payload changes. Across all settings, our adaptive controller consistently outperformed the controller in tracking body height and velocity commands, demonstrating enhanced robustness and adaptability without requiring explicit gait design or manual tuning.

Figures (6)

• Figure 1: Domain Randomization vs. Adaptive RL: While domain randomization is robust but conservative, our adaptive approach enables flexible locomotion with improved height and velocity tracking across challenging terrains.
• Figure 2: Overview of the proposed framework - History of observations is encoded to get a latent vector and body velocity using CE Net. The nominal policy and critic are trained in Phase 1, while the adaptive policy and adaptive critic are introduced in Phase 2 to enhance adaptation to payload variations. The combined action enables robust locomotion across diverse payload conditions.
• Figure 3: Adaptation of the quadruped robot to varying payload on stairs. (Top) The payload mass profile with 6 phases indicating mass transitions. (Middle) Norm of net contact forces over time. (Bottom) Norm of adaptive actions, demonstrating the controller’s response to mass changes and terrain transitions. Snapshots (1–6) depict representative instances during the locomotion sequence. (2), (3), (4) and (5) show how the robot has recovered from heavy payload change by generating higher GRFs. The graphs also show a positive correlation between the norm of the adapt action and the net contact forces.
• Figure 4: Performance comparison of baseline and adaptive controllers on flat terrain. The top section shows the payload mass profile. Both controllers exhibit similar velocity tracking and mean torque effort, while the adaptive controller achieves significantly better height tracking.
• Figure 5: Performance comparison of baseline vs adaptive controllers on stairs. The payload mass profile is shown at the top. The flat red segments in the curves indicate instances where the baseline controller fails to adjust to sudden payload changes, causing the robot to come to a complete halt. In contrast, the adaptive controller maintains stable locomotion, achieving better velocity and height tracking while significantly reducing height tracking error and torque effort across different payload conditions.