Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

TL;DR

Genie Sim 3.0 tackles the bottlenecks in robot learning by providing a high-fidelity, open-source simulation platform that enables large-scale data collection and automated, multi-dimensional evaluation. The approach integrates an LLM-driven scene generator, an LLM-based benchmark for automated instruction and evaluation, and a high-fidelity reconstruction pipeline to produce realistic environments, paired with teleoperation and automated data collection. It releases over 5140 assets, 10,000+ hours of synthetic data across 200 tasks, and 100,000 evaluation scenarios, demonstrating strong sim-to-real transfer under controlled conditions and enabling scalable policy training. This platform has practical impact by reducing reliance on physical hardware, accelerating model iteration, and offering reproducible, comprehensive diagnostics for embodied AI.

Abstract

The development of robust and generalizable robot learning models is critically contingent upon the availability of large-scale, diverse training data and reliable evaluation benchmarks. Collecting data in the physical world poses prohibitive costs and scalability challenges, and prevailing simulation benchmarks frequently suffer from fragmentation, narrow scope, or insufficient fidelity to enable effective sim-to-real transfer. To address these challenges, we introduce Genie Sim 3.0, a unified simulation platform for robotic manipulation. We present Genie Sim Generator, a large language model (LLM)-powered tool that constructs high-fidelity scenes from natural language instructions. Its principal strength resides in rapid and multi-dimensional generalization, facilitating the synthesis of diverse environments to support scalable data collection and robust policy evaluation. We introduce the first benchmark that pioneers the application of LLM for automated evaluation. It leverages LLM to mass-generate evaluation scenarios and employs Vision-Language Model (VLM) to establish an automated assessment pipeline. We also release an open-source dataset comprising more than 10,000 hours of synthetic data across over 200 tasks. Through systematic experimentation, we validate the robust zero-shot sim-to-real transfer capability of our open-source dataset, demonstrating that synthetic data can server as an effective substitute for real-world data under controlled conditions for scalable policy training. For code and dataset details, please refer to: https://github.com/AgibotTech/genie_sim.

Figures (9)

• Figure 2: The Automated Workflow of Genie Sim Generator. This module captures user intent via multi-round conversation, translates it into executable Python code, and compiles the final scene graph with assets for Isaac Sim.
• Figure 3: VLM-Driven Evaluation.
• Figure 4: Automated Data Collection. A complete task parsing and execution pipeline, which improves task success rate through waypoint filtering and a robust retry mechanism.
• Figure 5: Task Distribution Matrix. The dataset is constructed along three dimensions: Manipulation skill, Cognitive comprehension, and Task complexity.
• Figure 6: Sim to Real Experiments. Comparison of initial task setup between real and sim testing environments