Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control

Weisheng Xu, Qiwei Wu, Jiaxi Zhang, Tan Jing, Yangfan Li, Yuetong Fang, Jiaqi Xiong, Kai Wu, Rong Ou, Renjing Xu

TL;DR

This work proposes a closed-loop automated motion data generation and iterative framework that can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more.

Abstract

Physics-based humanoid control relies on training with motion datasets that have diverse data distributions. However, the fixed difficulty distribution of datasets limits the performance ceiling of the trained control policies. Additionally, the method of acquiring high-quality data through professional motion capture systems is constrained by costs, making it difficult to achieve large-scale scalability. To address these issues, we propose a closed-loop automated motion data generation and iterative framework. It can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more. Furthermore, our framework enables difficulty iteration of policies and data through physical metrics and objective evaluations, allowing the trained tracker to break through its original difficulty limits. On the PHC single-primitive tracker, using only approximately 1/10 of the AMASS dataset size, the average failure rate on the test set (2201 clips) is reduced by 45\% compared to the baseline. Finally, we conduct comprehensive ablation and comparative experiments to highlight the rationality and advantages of our framework.

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control

TL;DR

This work proposes a closed-loop automated motion data generation and iterative framework that can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more.

Abstract

Physics-based humanoid control relies on training with motion datasets that have diverse data distributions. However, the fixed difficulty distribution of datasets limits the performance ceiling of the trained control policies. Additionally, the method of acquiring high-quality data through professional motion capture systems is constrained by costs, making it difficult to achieve large-scale scalability. To address these issues, we propose a closed-loop automated motion data generation and iterative framework. It can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more. Furthermore, our framework enables difficulty iteration of policies and data through physical metrics and objective evaluations, allowing the trained tracker to break through its original difficulty limits. On the PHC single-primitive tracker, using only approximately 1/10 of the AMASS dataset size, the average failure rate on the test set (2201 clips) is reduced by 45\% compared to the baseline. Finally, we conduct comprehensive ablation and comparative experiments to highlight the rationality and advantages of our framework.
Paper Structure (58 sections, 8 equations, 16 figures, 11 tables, 1 algorithm)

This paper contains 58 sections, 8 equations, 16 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Humanoid Motion Datasets.
  4. Physics-Based Humanoid Control.
  5. Iterative Data Generation.
  6. Methodology
  7. Defining High-Difficulty Motion Data
  8. Dataset Generation
  9. Training and evaluation of Controller
  10. Competitive iteration between the controller and the dataset
  11. Automation of the iterative process
  12. Experiments
  13. Experimental Setup
  14. Motion professionalism verification
  15. Motion dataset features.
  16. ...and 43 more sections

Figures (16)

  • Figure 1: Overview of the CLAIMS pipeline: a closed-loop system that refines prompts from a 5-domain library (martial arts, dance, combat, sports, gymnastics), synthesizes motions with MDM, filters them by physics and VLM checks, trains humanoid trackers with RL, and uses multimodal feedback to generate progressively harder tasks.
  • Figure 2: Difficulty-aware variable library across five domains and four compositional axes.
  • Figure 3: Prompt-to-prompt data generation with Chain of Thought(Gemini).
  • Figure 4: Competitive iteration between the controller and the dataset
  • Figure 5: Schematic Diagram of the Automated Iterative Loop
  • ...and 11 more figures