Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Whole-body Humanoid Robot Locomotion with Human Reference

Qiang Zhang, Peter Cui, David Yan, Jingkai Sun, Yiqun Duan, Gang Han, Wen Zhao, Weining Zhang, Yijie Guo, Arthur Zhang, Renjing Xu

TL;DR

This work introduces Adam, a full-size humanoid robot, and a novel imitation-learning framework using adversarial motion priors guided by human motion data to overcome reward-design and sim-to-real challenges in humanoid locomotion. The approach combines AMP-based adversarial learning with end-to-end PPO reinforcement learning, employing world-coordinate rewards, detailed periodic gait incentives, and extensive domain randomization to achieve robust, human-like locomotion across unseen terrains. Experimental validation spans Webots, Isaac Gym, and real hardware, demonstrating cross-platform consistency and real-world viability, including previously unseen capabilities like heel-to-toe running and straight-knee arm swing. The study advances practical humanoid locomotion by reducing manual reward engineering and enabling scalable, perception-augmented future enhancements for autonomous mobility.

Abstract

Recently, humanoid robots have made significant advances in their ability to perform challenging tasks due to the deployment of Reinforcement Learning (RL), however, the inherent complexity of humanoid robots, including the difficulty of designing complicated reward functions and training entire sophisticated systems, still poses a notable challenge. To conquer these challenges, after many iterations and in-depth investigations, we have meticulously developed a full-size humanoid robot, "Adam", whose innovative structural design greatly improves the efficiency and effectiveness of the imitation learning process. In addition, we have developed a novel imitation learning framework based on an adversarial motion prior, which applies not only to Adam but also to humanoid robots in general. Using the framework, Adam can exhibit unprecedented human-like characteristics in locomotion tasks. Our experimental results demonstrate that the proposed framework enables Adam to achieve human-comparable performance in complex locomotion tasks, marking the first time that human locomotion data has been used for imitation learning in a full-size humanoid robot.

Whole-body Humanoid Robot Locomotion with Human Reference

TL;DR

This work introduces Adam, a full-size humanoid robot, and a novel imitation-learning framework using adversarial motion priors guided by human motion data to overcome reward-design and sim-to-real challenges in humanoid locomotion. The approach combines AMP-based adversarial learning with end-to-end PPO reinforcement learning, employing world-coordinate rewards, detailed periodic gait incentives, and extensive domain randomization to achieve robust, human-like locomotion across unseen terrains. Experimental validation spans Webots, Isaac Gym, and real hardware, demonstrating cross-platform consistency and real-world viability, including previously unseen capabilities like heel-to-toe running and straight-knee arm swing. The study advances practical humanoid locomotion by reducing manual reward engineering and enabling scalable, perception-augmented future enhancements for autonomous mobility.

Abstract

Recently, humanoid robots have made significant advances in their ability to perform challenging tasks due to the deployment of Reinforcement Learning (RL), however, the inherent complexity of humanoid robots, including the difficulty of designing complicated reward functions and training entire sophisticated systems, still poses a notable challenge. To conquer these challenges, after many iterations and in-depth investigations, we have meticulously developed a full-size humanoid robot, "Adam", whose innovative structural design greatly improves the efficiency and effectiveness of the imitation learning process. In addition, we have developed a novel imitation learning framework based on an adversarial motion prior, which applies not only to Adam but also to humanoid robots in general. Using the framework, Adam can exhibit unprecedented human-like characteristics in locomotion tasks. Our experimental results demonstrate that the proposed framework enables Adam to achieve human-comparable performance in complex locomotion tasks, marking the first time that human locomotion data has been used for imitation learning in a full-size humanoid robot.
Paper Structure (17 sections, 12 equations, 7 figures, 3 tables)

This paper contains 17 sections, 12 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Legged robots locomotion
  4. Learning from Human Reference
  5. Preliminary
  6. The Structure of Humanoid Robot Adam
  7. Motion Capture and Re-Targeting
  8. Reinforcement Learning on Legged Robots
  9. Method
  10. Learning with Human Reference by Adversarial Motion Priors
  11. End-to-End Reinforcement Learning on Humanoid Robot
  12. Experiments
  13. Training and Implementation Details
  14. Training Setup
  15. Domian Randomization
  16. ...and 2 more sections

Figures (7)

  • Figure 1: The top image displays the humanoid robot Adam walking on unseen terrain, and the bottom image shows Adam moving from a standing position to running.
  • Figure 2: Schematic representation of the humanoid robot Adam(Lite), showing both the front view and the angled side view. Particularly, the design of its hip joint distinctly emulates the human skeleton, demonstrating a remarkable level of anthropomorphism throughout its structure.
  • Figure 3: Adversarial Motion Priors Imitation Training Framework of Humanoid Robot
  • Figure 4: Visualization of periodic rewards based on the von Mises distribution
  • Figure 5: Real Robot Experiments. We tested our method on Adam. (A)(B) demonstrate the robot's robust locomotion performance under external disturbances. Especially, it demonstrated the characteristic of "straight-knee" (C)(D) show the human-like swinging of the upper limbs. (E)(F) display the robustness and human-likeness on unknown and complex terrains. (G)(H) present for the first time the humanoid robot's "heel-to-toe" running and walking gaits.
  • ...and 2 more figures