McARL:Morphology-Control-Aware Reinforcement Learning for Generalizable Quadrupedal Locomotion

TL;DR

McARL tackles the challenge of generalizing quadrupedal locomotion across morphologies by learning a morphology-aware policy. It introduces a 14-D morphology vector and a morphology encoder to produce a latent $z_m$ that conditions both the actor and critic within a PPO framework, augmented by History-Aware Curriculum Learning to leverage history in curriculum design. The method demonstrates zero-shot transfer from a single trained morph to Go2, Mini Cheetah, and A1 in simulation, with up to 3.5 m/s transferred velocity and 6.0 m/s on the trained Go1, along with a real-world deployment on Go1 after minimal fine-tuning. Across baselines, McARL achieves 44–150% higher transfer performance on unseen morphologies, highlighting the value of explicit morphology conditioning and history-informed curricula for robust, generalizable legged locomotion.

Abstract

We present Morphology-Control-Aware Reinforcement Learning (McARL), a new approach to overcome challenges of hyperparameter tuning and transfer loss, enabling generalizable locomotion across robot morphologies. We use a morphology-conditioned policy by incorporating a randomized morphology vector, sampled from a defined morphology range, into both the actor and critic networks. This allows the policy to learn parameters that generalize to robots with similar characteristics. We demonstrate that a single policy trained on a Unitree Go1 robot using McARL can be transferred to a different morphology (e.g., Unitree Go2 robot) and can achieve zero-shot transfer velocity of up to 3.5 m/s without retraining or fine-tuning. Moreover, it achieves 6.0 m/s on the training Go1 robot and generalizes to other morphologies like A1 and Mini Cheetah. We also analyze the impact of morphology distance on transfer performance and highlight McARL's advantages over prior approaches. McARL achieves 44-150% higher transfer performance on Go2, Mini Cheetah, and A1 compared to PPO variants.

Figures (5)

• Figure 1: Overview of our Morphology-Control-Aware Reinforcement Learning (McARL), a novel framework incorporates 14-dimensional morphology vector into the PPO policy.
• Figure 2: Morphology embedded policy controller testing in simulator on go1 robot (row 1) and Sim-to-sim zero shot transfer on go2, a1 and mini cheetah (row 2, 3, 4). We also perform real world testing on go1 robot for command velocity of 1m/sec (Last row).
• Figure 3: The first image of each row represents on which ronot McARL has been trained and the remaining following images represent the transfer velocities on the other robots. The Go1 robot (first row) achieves great velocity of 6m/s with max transfer on go2 at around 3.5m/s, the Go2 (second row) robot reaches 3.5 m/s and similar transfer rate for go1, while the transfer loss starts to get bad from Mini Cheetah robot (third row) and worse for the A1 robot (last row). Interstingly the A1 robots performance was not that stable and more of galloping behavior indicating the need for fine tuning the config parameters and tuning the rewards accordingly.
• Figure 4: Parallelized training for go1, go2, mini cheetah and A1 robot (total 4000 environments) in Isaac Gym simulator (Top row). Transfer loss and the distance correlation for McARL, as the morphology between the trained morphology and the unseen morphology increases, so does the transfer losses.
• Figure 5: PPO combination of P3 PPO + Morphology (without control params) + HACL performs overall best compared to other combinations like P0, P1, P2, P4, P5 and has pretty decent transfer over all other morphologies like Go1, Go2, A1 and mini cheetah robots. (Please refer to appendix for detailed analysis)