Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GigaBrain-0: A World Model-Powered Vision-Language-Action Model

GigaBrain Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, Peng Li, Qiuping Deng, Runqi Ouyang, Wenkang Qin, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yilong Li, Yiran Ding, Yuan Xu, Yun Ye, Yukun Zhou, Zhehao Dong, Zhenan Wang, Zhichao Liu, Zheng Zhu

TL;DR

The paper tackles the high data costs of training Vision-Language-Action robots by introducing GigaBrain-0, a foundation model powered by world-model-generated data (Real2Real, view transfer, sim2real, and human transfer) to improve generalization. It integrates RGB-D perception and embodied Chain-of-Thought within a mixture-of-transformers architecture, along with Knowledge Insulation to harmonize semantic reasoning and action learning. The approach is validated through extensive real-world experiments across dexterous, long-horizon, and mobile manipulation tasks, showing superior generalization to appearance, placement, and viewpoint variations. An edge-friendly variant, GigaBrain-0-Small, demonstrates practical on-device deployment on platforms like NVIDIA Jetson AGX Orin, highlighting the model’s real-time viability.

Abstract

Training Vision-Language-Action (VLA) models for generalist robots typically requires large-scale real-world robot data, which is expensive and time-consuming to collect. The inefficiency of physical data collection severely limits the scalability, and generalization capacity of current VLA systems. To address this challenge, we introduce GigaBrain-0, a novel VLA foundation model empowered by world model-generated data (e.g., video generation, real2real transfer, human transfer, view transfer, sim2real transfer data). By leveraging world models to generate diverse data at scale, GigaBrain-0 significantly reduces reliance on real robot data while improving cross-task generalization. Our approach further improves policy robustness through RGBD input modeling and embodied Chain-of-Thought (CoT) supervision, enabling the model to reason about spatial geometry, object states, and long-horizon dependencies during task execution. This leads to substantial gains in real-world performance on dexterous, long-horizon, and mobile manipulation tasks. Extensive experiments demonstrate that GigaBrain-0 achieves superior generalization across variations in appearances (e.g., textures, colors), object placements, and camera viewpoints. Additionally, we present GigaBrain-0-Small, an optimized lightweight variant designed to run efficiently on devices such as the NVIDIA Jetson AGX Orin.

GigaBrain-0: A World Model-Powered Vision-Language-Action Model

TL;DR

The paper tackles the high data costs of training Vision-Language-Action robots by introducing GigaBrain-0, a foundation model powered by world-model-generated data (Real2Real, view transfer, sim2real, and human transfer) to improve generalization. It integrates RGB-D perception and embodied Chain-of-Thought within a mixture-of-transformers architecture, along with Knowledge Insulation to harmonize semantic reasoning and action learning. The approach is validated through extensive real-world experiments across dexterous, long-horizon, and mobile manipulation tasks, showing superior generalization to appearance, placement, and viewpoint variations. An edge-friendly variant, GigaBrain-0-Small, demonstrates practical on-device deployment on platforms like NVIDIA Jetson AGX Orin, highlighting the model’s real-time viability.

Abstract

Training Vision-Language-Action (VLA) models for generalist robots typically requires large-scale real-world robot data, which is expensive and time-consuming to collect. The inefficiency of physical data collection severely limits the scalability, and generalization capacity of current VLA systems. To address this challenge, we introduce GigaBrain-0, a novel VLA foundation model empowered by world model-generated data (e.g., video generation, real2real transfer, human transfer, view transfer, sim2real transfer data). By leveraging world models to generate diverse data at scale, GigaBrain-0 significantly reduces reliance on real robot data while improving cross-task generalization. Our approach further improves policy robustness through RGBD input modeling and embodied Chain-of-Thought (CoT) supervision, enabling the model to reason about spatial geometry, object states, and long-horizon dependencies during task execution. This leads to substantial gains in real-world performance on dexterous, long-horizon, and mobile manipulation tasks. Extensive experiments demonstrate that GigaBrain-0 achieves superior generalization across variations in appearances (e.g., textures, colors), object placements, and camera viewpoints. Additionally, we present GigaBrain-0-Small, an optimized lightweight variant designed to run efficiently on devices such as the NVIDIA Jetson AGX Orin.
Paper Structure (15 sections, 1 equation, 19 figures, 2 tables)

This paper contains 15 sections, 1 equation, 19 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Vision-Language-Action Models
  4. World Models as Data Engines
  5. GigaBrain-0 Model
  6. GigaBrain-0 Data
  7. Real-World Data
  8. World-Model-Generated Data
  9. Experiment
  10. Dexterous Manipulation Experiment
  11. Long-horizon Experiment
  12. Mobile Manipulation Experiment
  13. Manipulation Generalization Experiment
  14. On-device Experiment
  15. Conclusion and Future Work

Figures (19)

  • Figure 1:
  • Figure 2: GigaBrain-0's self-collected real-world robot data is gathered from PiPER arms and the AgiBot G1 platform, spanning diverse environments including homes, supermarkets, factories, and office settings.
  • Figure 3: GigaWorld enables Real2Real apperance transfer by taking real-world captured data and generating generalized variations in texture, color, lighting, and material properties.
  • Figure 4: GigaWorld supports view transfer by re-rendering real-world captured data from diverse viewpoints, thereby enriching the dataset with varied perspective changes.
  • Figure 5: GigaWorld enables Sim2Real transfer by generalizing simulation-collected data in terms of texture, color, lighting, and material properties to better bridge the domain gap and enhance realism.
  • ...and 14 more figures