Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Track Any Motions under Any Disturbances

Zhikai Zhang, Jun Guo, Chao Chen, Jilong Wang, Chenghuai Lin, Yunrui Lian, Han Xue, Zhenrong Wang, Maoqi Liu, Jiangran Lyu, Huaping Liu, He Wang, Li Yi

TL;DR

This work addresses the challenge of tracking diverse, highly dynamic humanoid motions under real-world disturbances. It introduces Any2Track, a two-stage RL framework: AnyTracker learns a general motion-tracking policy with canonicalized action spaces and specialist-to-generalist distillation, and AnyAdapter provides online dynamics adaptability via a dynamics-aware world model and a LoRA-like adapter, all trained to preserve motion expressiveness. The approach achieves zero-shot sim2real transfer on Unitree G1 and demonstrates superior tracking and disturbance adaptation in both simulation and real-world tests, across terrains, external forces, and payload changes. This framework advances practical humanoid motion tracking toward open-world, disturbance-rich environments and enables robust downstream applications like tele-operation and skill learning.

Abstract

A foundational humanoid motion tracker is expected to be able to track diverse, highly dynamic, and contact-rich motions. More importantly, it needs to operate stably in real-world scenarios against various dynamics disturbances, including terrains, external forces, and physical property changes for general practical use. To achieve this goal, we propose Any2Track (Track Any motions under Any disturbances), a two-stage RL framework to track various motions under multiple disturbances in the real world. Any2Track reformulates dynamics adaptability as an additional capability on top of basic action execution and consists of two key components: AnyTracker and AnyAdapter. AnyTracker is a general motion tracker with a series of careful designs to track various motions within a single policy. AnyAdapter is a history-informed adaptation module that endows the tracker with online dynamics adaptability to overcome the sim2real gap and multiple real-world disturbances. We deploy Any2Track on Unitree G1 hardware and achieve a successful sim2real transfer in a zero-shot manner. Any2Track performs exceptionally well in tracking various motions under multiple real-world disturbances.

Track Any Motions under Any Disturbances

TL;DR

This work addresses the challenge of tracking diverse, highly dynamic humanoid motions under real-world disturbances. It introduces Any2Track, a two-stage RL framework: AnyTracker learns a general motion-tracking policy with canonicalized action spaces and specialist-to-generalist distillation, and AnyAdapter provides online dynamics adaptability via a dynamics-aware world model and a LoRA-like adapter, all trained to preserve motion expressiveness. The approach achieves zero-shot sim2real transfer on Unitree G1 and demonstrates superior tracking and disturbance adaptation in both simulation and real-world tests, across terrains, external forces, and payload changes. This framework advances practical humanoid motion tracking toward open-world, disturbance-rich environments and enables robust downstream applications like tele-operation and skill learning.

Abstract

A foundational humanoid motion tracker is expected to be able to track diverse, highly dynamic, and contact-rich motions. More importantly, it needs to operate stably in real-world scenarios against various dynamics disturbances, including terrains, external forces, and physical property changes for general practical use. To achieve this goal, we propose Any2Track (Track Any motions under Any disturbances), a two-stage RL framework to track various motions under multiple disturbances in the real world. Any2Track reformulates dynamics adaptability as an additional capability on top of basic action execution and consists of two key components: AnyTracker and AnyAdapter. AnyTracker is a general motion tracker with a series of careful designs to track various motions within a single policy. AnyAdapter is a history-informed adaptation module that endows the tracker with online dynamics adaptability to overcome the sim2real gap and multiple real-world disturbances. We deploy Any2Track on Unitree G1 hardware and achieve a successful sim2real transfer in a zero-shot manner. Any2Track performs exceptionally well in tracking various motions under multiple real-world disturbances.

Paper Structure

This paper contains 28 sections, 4 equations, 4 figures, 6 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. Motion Tracking
  4. Online Dynamics Adaptation for Legged Robots
  5. TRACK ANY MOTIONS UNDER ANY DISTURBANCE
  6. AnyTracker: Track Any Motions
  7. Canonicalized Action Spaces
  8. Motion Clustering and Specialist-to-Generalist
  9. AnyAdapter: Adapt to Any Disturbances
  10. Learn Informative Dynamics Embeddings
  11. Dynamics Adaptability Injection
  12. EXPERIMENTS
  13. Results of Tracking Any Motions
  14. Experiment Setting
  15. Experiment Metrics
  16. ...and 13 more sections

Figures (4)

  • Figure 1: (a) The humanoid tracks diverse, highly dynamic, and contact-rich motions using a single policy. (b) The humanoid tracks motions on different terrains. (c) The humanoid tracks motions against different external forces, including pulling forces from the rope and pushing forces from the human feet. (d) The humanoid tracks motions against physical property changes (the payload on the back). (e) The humanoid tracks motions against all dynamics disturbances, including terrains, external forces, and physical property changes simultaneously.
  • Figure 2: Overview of our method.Any2Track consists of two key components: AnyTracker and AnyAdapter. AnyTracker is a general motion tracker with a series of careful designs. AnyAdapter is a history-informed adaptation module on top of AnyTracker. AnyAdapter endows the base tracker with online dynamics adaptability without compromising its fundamental expressive motion tracking capability.
  • Figure 3: Overview of methods evaluated in online dynamics adaptation experiments. All algorithms are trained with asymmetric PPO, and the critics are omitted in the figure.
  • Figure 4: Different environment disturbance settings in the real-world experiment.