RoboMIND 2.0: A Multimodal, Bimanual Mobile Manipulation Dataset for Generalizable Embodied Intelligence

TL;DR

RoboMIND 2.0 furnishes a large-scale, multimodal, multi-embodiment bimanual manipulation resource, pairing real-world trajectories with tactile and mobile data and digital twins to enable robust sim-to-real learning. It introduces the MIND-2 dual-system framework, combining a slow Vision-Language Model planner with a fast Vision-Language-Action policy trained offline via Implicit Q-Learning, to tackle long-horizon manipulation across diverse morphologies. The dataset spans 759 tasks, 1,139 objects, six embodiments, and includes 12K tactile sequences plus 20K simulated trajectories, offering unprecedented diversity and facilitating cross-embodiment generalization, tactile-enabled dexterity, and open-world policy learning. Empirical results show 3D-aware imitation learning and cross-embodiment VLA models like XR-1 achieve strong performance, while MIND-2 significantly outperforms single-task baselines on long-horizon collaborative tasks, highlighting the value of large-scale multimodal data and offline RL for embodied AI.

Abstract

While data-driven imitation learning has revolutionized robotic manipulation, current approaches remain constrained by the scarcity of large-scale, diverse real-world demonstrations. Consequently, the ability of existing models to generalize across long-horizon bimanual tasks and mobile manipulation in unstructured environments remains limited. To bridge this gap, we present RoboMIND 2.0, a comprehensive real-world dataset comprising over 310K dual-arm manipulation trajectories collected across six distinct robot embodiments and 739 complex tasks. Crucially, to support research in contact-rich and spatially extended tasks, the dataset incorporates 12K tactile-enhanced episodes and 20K mobile manipulation trajectories. Complementing this physical data, we construct high-fidelity digital twins of our real-world environments, releasing an additional 20K-trajectory simulated dataset to facilitate robust sim-to-real transfer. To fully exploit the potential of RoboMIND 2.0, we propose MIND-2 system, a hierarchical dual-system frame-work optimized via offline reinforcement learning. MIND-2 integrates a high-level semantic planner (MIND-2-VLM) to decompose abstract natural language instructions into grounded subgoals, coupled with a low-level Vision-Language-Action executor (MIND-2-VLA), which generates precise, proprioception-aware motor actions.

Figures (14)

• Figure 1: Overview of the RoboMIND 2.0. We introduce RoboMIND 2.0, a large-scale dataset comprising 310K dual-arm trajectories collected from six heterogeneous robot embodiments, totaling over 1,000 hours. The dataset features rich modalities, including 12K tactile-enriched sequences and 20K mobile manipulation trajectories. Collected through a unified teleoperation and quality assurance pipeline, RoboMIND 2.0 ensures consistent proprioception and provides fine-grained natural language annotations. To support scalable training and evaluation, we release digital-twin USD assets and 20K simulation trajectories aligned with real-world tasks. Building on this foundation, we propose MIND-2, a dual-system controller that integrates a slow high-level planner MIND-2-VLM with a fast low-level policy MIND-2-VLA, enabling robust long-horizon mobile manipulation across diverse scenarios.
• Figure 2: Collection platform of Franka and UR5e. Collect a robotic manipulation dataset by controlling the dual-arm system (Franka and UR5e) via HACTS.
• Figure 3: Collection platform of AgileX and ARX. We use a VR headset to control the ARK robot for data collection, and employ a slave arm to teleoperate the master arm for gathering the Agliex manipulation dataset.
• Figure 4: Visualization of Tian Yi and Tien Kung.
• Figure 5: Illustration of the twelve data–inspection categories used in our quality-control workflow, including unintended contact, motion irregularities, sensing artifacts, and task-level execution failures.