Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

General Humanoid Whole-Body Control via Pretraining and Fast Adaptation

Zepeng Wang, Jiangxing Wang, Shiqing Yao, Yu Zhang, Ziluo Ding, Ming Yang, Yuxuan Wang, Haobin Jiang, Chao Ma, Xiaochuan Shi, Zongqing Lu

TL;DR

FAST addresses the challenge of general humanoid whole-body control under distribution shifts by coupling a pretrained, Center-of-Mass-Aware base controller with a lightweight Parseval-regularized residual policy. The residual is trained with KL constraints to bound drift, and action composition follows $a_t = a_t^b + a_t^r$, enabling rapid adaptation while preserving prior behavior. The framework relies on curated motion datasets and a mixture-of-experts policy, augmented with CoM/CoP signals to improve balance during tracking and teleoperation. Empirical results in simulation and on a real Unitree G1 robot show that FAST outperforms state-of-the-art baselines in robustness, adaptation efficiency, and generalization, including zero-shot high-dynamic tracking, fast adaptation to low-quality motions, and real-time teleoperation.

Abstract

Learning a general whole-body controller for humanoid robots remains challenging due to the diversity of motion distributions, the difficulty of fast adaptation, and the need for robust balance in high-dynamic scenarios. Existing approaches often require task-specific training or suffer from performance degradation when adapting to new motions. In this paper, we present FAST, a general humanoid whole-body control framework that enables Fast Adaptation and Stable Motion Tracking. FAST introduces Parseval-Guided Residual Policy Adaptation, which learns a lightweight delta action policy under orthogonality and KL constraints, enabling efficient adaptation to out-of-distribution motions while mitigating catastrophic forgetting. To further improve physical robustness, we propose Center-of-Mass-Aware Control, which incorporates CoM-related observations and objectives to enhance balance when tracking challenging reference motions. Extensive experiments in simulation and real-world deployment demonstrate that FAST consistently outperforms state-of-the-art baselines in robustness, adaptation efficiency, and generalization.

General Humanoid Whole-Body Control via Pretraining and Fast Adaptation

TL;DR

FAST addresses the challenge of general humanoid whole-body control under distribution shifts by coupling a pretrained, Center-of-Mass-Aware base controller with a lightweight Parseval-regularized residual policy. The residual is trained with KL constraints to bound drift, and action composition follows , enabling rapid adaptation while preserving prior behavior. The framework relies on curated motion datasets and a mixture-of-experts policy, augmented with CoM/CoP signals to improve balance during tracking and teleoperation. Empirical results in simulation and on a real Unitree G1 robot show that FAST outperforms state-of-the-art baselines in robustness, adaptation efficiency, and generalization, including zero-shot high-dynamic tracking, fast adaptation to low-quality motions, and real-time teleoperation.

Abstract

Learning a general whole-body controller for humanoid robots remains challenging due to the diversity of motion distributions, the difficulty of fast adaptation, and the need for robust balance in high-dynamic scenarios. Existing approaches often require task-specific training or suffer from performance degradation when adapting to new motions. In this paper, we present FAST, a general humanoid whole-body control framework that enables Fast Adaptation and Stable Motion Tracking. FAST introduces Parseval-Guided Residual Policy Adaptation, which learns a lightweight delta action policy under orthogonality and KL constraints, enabling efficient adaptation to out-of-distribution motions while mitigating catastrophic forgetting. To further improve physical robustness, we propose Center-of-Mass-Aware Control, which incorporates CoM-related observations and objectives to enhance balance when tracking challenging reference motions. Extensive experiments in simulation and real-world deployment demonstrate that FAST consistently outperforms state-of-the-art baselines in robustness, adaptation efficiency, and generalization.
Paper Structure (35 sections, 2 theorems, 21 equations, 6 figures, 6 tables)

This paper contains 35 sections, 2 theorems, 21 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learning-based Humanoid Whole-Body Control
  4. Fast Adaptation for Humanoid Control
  5. Method
  6. Motion Dataset Curation
  7. General Whole-Body Controller
  8. Parseval-Guided Residual Policy Adaptation
  9. Experiments
  10. Experiment Setup
  11. Comparison with Existing Methods
  12. Evaluation of Fast Adaptation with Performance Preservation
  13. Ablation Studies
  14. Ablations on CoM-Aware Control
  15. Ablations on Fast Adaptation
  16. ...and 20 more sections

Key Result

Proposition 3.1

Consider a residual policy $\pi_r:\mathbb{R}^d \rightarrow \mathbb{R}^m$ parameterized by an $L$-layer feed-forward MLP with 1-Lipschitz activation functions $\phi(\cdot)$. Suppose Parseval regularization is applied such that for each layer $l=1,\ldots,L-1$, the weight matrix satisfies $\| W_l^\top

Figures (6)

  • Figure 1: FAST is a general and fast-adaptation framework for humanoid whole-body control. It enables zero-shot high-dynamic motion tracking and real-time teleoperation with strong robustness and balance beyond simulation. For low-quality or out-of-distribution motion references, FAST further supports lightweight residual adaptation for rapid and stable specialization.
  • Figure 2: An overview of FAST. Our framework consists of three stages. (1) We construct a curated humanoid motion dataset via human-to-humanoid retargeting with auxiliary physical signals. (2) We train a general whole-body controller with a Mixture-of-Experts architecture and Center-of-Mass-Aware control. (3) We perform fast adaptation via Parseval-guided residual policy learning.
  • Figure 3: Fast adaptation on LaFan1 and MotionX (target) with performance retention on AMASS (source).
  • Figure 4: Visualizations of representative motions comparing CoM-Aware Control with the baseline. (a) Forward bending. (b) High knee lift. (c) High leg lift. (d) Side-to-side swaying gait with turning. CoM-Aware Control consistently maintains balance and reduces instability, while the baseline exhibits drift or falls.
  • Figure 5: Zero-shot generalization to high-dynamic motions on a Unitree G1.
  • ...and 1 more figures

Theorems & Definitions (2)

  • Proposition 3.1
  • Proposition 3.2