TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

TL;DR

TWIST2 delivers a portable, mocap-free humanoid data collection and teleoperation system that preserves full-body control through a hierarchical framework and egocentric vision. By integrating a lightweight neck, PICO-based motion capture, and a real-time retargeting pipeline, it enables scalable, long-horizon dexterous and mobile tasks with a single operator. A Diffusion-based visuomotor policy is trained on large demonstration datasets to autonomously drive the full humanoid, achieving tasks such as whole-body dexterous manipulation and Kick-T. The work emphasizes reproducibility and open science, offering open-source code and data to foster scalable, standardized research in humanoid robotics.

Abstract

Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effective data collection frameworks. Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. Our system leverages PICO4U VR for obtaining real-time whole-body human motions, with a custom 2-DoF robot neck (cost around $250) for egocentric vision, enabling holistic human-to-humanoid control. We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate. Building on this pipeline, we propose a hierarchical visuomotor policy framework that autonomously controls the full humanoid body based on egocentric vision. Our visuomotor policy successfully demonstrates whole-body dexterous manipulation and dynamic kicking tasks. The entire system is fully reproducible and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also open-sourced at https://twist-data.github.io .

Figures (14)

• Figure 1: System overview of TWIST2. We build a holistic humanoid teleoperation system with portable devices and egocentric active vision, enabling scalable imitation data collection. With data collected, we build a hierarchical visuomotor policy learning framework that directly predicts whole-body joint positions.
• Figure 2: TWIST2 Neck. We design a simple yet effective 2-DoF neck that can be easily assembled for a non-expert user and can be attached/detached to/from a Unitree G1 without removing the original LiDAR.
• Figure 3: TWIST2 Neck in MuJoCo. To facilitate the research in simulation and standardize our data, we build MuJoCo XML files for our TWIST2 neck.
• Figure 4: Mimic the human neck with the robot neck. We found that a 2 DoFs neck (yaw and pitch) is sufficient to mimic major human neck movements.
• Figure 5: Mapping VR human bodies to robot links.