Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation

Yixue Zhang, Kun Wu, Zhi Gao, Zhen Zhao, Pei Ren, Zhiyuan Xu, Fei Liao, Xinhua Wang, Shichao Fan, Di Wu, Qiuxuan Feng, Meng Li, Zhengping Che, Chang Liu, Jian Tang

TL;DR

RoboGene addresses the real-world data bottleneck in robotic manipulation by autonomously generating diverse, physically grounded tasks for pre-training Vision-Language-Action models. It combines diversity-driven LFU sampling, a triad of self-reflection evaluators, and a memory-augmented refinement loop to ensure tasks are both novel and executable. Large-scale experiments show RoboGene outperforms state-of-the-art foundation models and human designs in task quality and grounding, while real-world validation demonstrates improved generalization in VLA policies. The framework advances scalable, high-quality data generation for embodied AI and provides open resources to support ongoing research.

Abstract

The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from web in vision or language, robotic data collection is an active process incurring prohibitive physical costs. Consequently, automated task curation to maximize data value remains a critical yet under-explored challenge. Existing manual methods are unscalable and biased toward common tasks, while off-the-shelf foundation models often hallucinate physically infeasible instructions. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots. RoboGene integrates three core components: diversity-driven sampling for broad task coverage, self-reflection mechanisms to enforce physical constraints, and human-in-the-loop refinement for continuous improvement. We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories and introducing novel metrics to assess task quality, feasibility, and diversity. Results demonstrate that RoboGene significantly outperforms state-of-the-art foundation models (e.g., GPT-4o, Gemini 2.5 Pro). Furthermore, real-world experiments show that VLA models pre-trained with RoboGene achieve higher success rates and superior generalization, underscoring the importance of high-quality task generation. Our project is available at https://robogene-boost-vla.github.io.

RoboGene: Boosting VLA Pre-training via Diversity-Driven Agentic Framework for Real-World Task Generation

TL;DR

RoboGene addresses the real-world data bottleneck in robotic manipulation by autonomously generating diverse, physically grounded tasks for pre-training Vision-Language-Action models. It combines diversity-driven LFU sampling, a triad of self-reflection evaluators, and a memory-augmented refinement loop to ensure tasks are both novel and executable. Large-scale experiments show RoboGene outperforms state-of-the-art foundation models and human designs in task quality and grounding, while real-world validation demonstrates improved generalization in VLA policies. The framework advances scalable, high-quality data generation for embodied AI and provides open resources to support ongoing research.

Abstract

The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from web in vision or language, robotic data collection is an active process incurring prohibitive physical costs. Consequently, automated task curation to maximize data value remains a critical yet under-explored challenge. Existing manual methods are unscalable and biased toward common tasks, while off-the-shelf foundation models often hallucinate physically infeasible instructions. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots. RoboGene integrates three core components: diversity-driven sampling for broad task coverage, self-reflection mechanisms to enforce physical constraints, and human-in-the-loop refinement for continuous improvement. We conduct extensive quantitative analysis and large-scale real-world experiments, collecting datasets of 18k trajectories and introducing novel metrics to assess task quality, feasibility, and diversity. Results demonstrate that RoboGene significantly outperforms state-of-the-art foundation models (e.g., GPT-4o, Gemini 2.5 Pro). Furthermore, real-world experiments show that VLA models pre-trained with RoboGene achieve higher success rates and superior generalization, underscoring the importance of high-quality task generation. Our project is available at https://robogene-boost-vla.github.io.
Paper Structure (24 sections, 1 equation, 8 figures, 6 tables, 1 algorithm)

This paper contains 24 sections, 1 equation, 8 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generalist Vision-Language-Action Models
  4. Automated Task Generation
  5. Methodology
  6. Overview
  7. Diversity-Driven Task Space Sampling
  8. Self-Reflection and Task Generation
  9. Integrating Human Feedback via Memory
  10. Experiments
  11. Individual Task Evaluation
  12. Dataset Diversity Analysis
  13. Real-World Manipulation Task Evaluation
  14. Conclusion
  15. Implementation Details of RoboGene
  16. ...and 9 more sections

Figures (8)

  • Figure 1: RoboGene is an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks, thereby enhancing the foundational capability and generalizability of VLA model pre-training.
  • Figure 2: Overview of RoboGene. The framework comprises three key components: (1) a diversity-driven sampling mechanism based on a Least Frequently Used (LFU) strategy; (2) a self-reflection mechanism that enhances task generation quality; and (3) a long-term memory module that consolidates Human-in-the-Loop (HITL) feedback.
  • Figure 3: Scenario diversity results across 8 pre-defined scenario categories. RoboGene demonstrates a highly balanced distribution.
  • Figure 4: Skill diversity results given a total inventory of 118 skills. RoboGene significantly outperforms baselines in skill coverage and demonstrates a highly balanced distribution.
  • Figure 5: (a) Usage frequency of the top-15 objects across the 900 tasks generated by each method. (b) Total object count for tasks generated by each method, given a total inventory of 1,137 objects.
  • ...and 3 more figures