InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy

TL;DR

InternVLA-M1 addresses the gap between language instructions and embodied action by introducing spatial grounding as a bridge in a dual-system Vision-Language-Action framework. It deploys a spatially guided two-stage training pipeline—spatial grounding pre-training for a VLM planner and spatially guided action post-training for embodiment-specific control—achieving robust generalization across simulation and real-world tasks. The approach is validated on public benchmarks (SimplerEnv, LIBERO), large-scale in-house simulations, and real-world cluttered scenes, with consistently strong gains over state-of-the-art baselines and notable improvements in unseen objects and long-horizon tasks. By leveraging a scalable synthetic data engine and co-training on diverse spatial grounding and manipulation data, InternVLA-M1 demonstrates the potential of spatial priors to enable scalable, resilient generalist robot policy learning and deployment.

Abstract

We introduce InternVLA-M1, a unified framework for spatial grounding and robot control that advances instruction-following robots toward scalable, general-purpose intelligence. Its core idea is spatially guided vision-language-action training, where spatial grounding serves as the critical link between instructions and robot actions. InternVLA-M1 employs a two-stage pipeline: (i) spatial grounding pre-training on over 2.3M spatial reasoning data to determine ``where to act'' by aligning instructions with visual, embodiment-agnostic positions, and (ii) spatially guided action post-training to decide ``how to act'' by generating embodiment-aware actions through plug-and-play spatial prompting. This spatially guided training recipe yields consistent gains: InternVLA-M1 outperforms its variant without spatial guidance by +14.6% on SimplerEnv Google Robot, +17% on WidowX, and +4.3% on LIBERO Franka, while demonstrating stronger spatial reasoning capability in box, point, and trace prediction. To further scale instruction following, we built a simulation engine to collect 244K generalizable pick-and-place episodes, enabling a 6.2% average improvement across 200 tasks and 3K+ objects. In real-world clustered pick-and-place, InternVLA-M1 improved by 7.3%, and with synthetic co-training, achieved +20.6% on unseen objects and novel configurations. Moreover, in long-horizon reasoning-intensive scenarios, it surpassed existing works by over 10%. These results highlight spatially guided training as a unifying principle for scalable and resilient generalist robots. Code and models are available at https://github.com/InternRobotics/InternVLA-M1.

Figures (12)

• Figure 1: InternVLA-M1 integrates spatial grounding into the vision–language–action training pipeline. Given a task instruction, the VLM planner produces latent plans through explicit spatial prompting, which then effectively guides the action expert to generate control signals.
• Figure 2: Overview of InternVLA-M1. InternVLA-M1 adopts a spatially guided two-stage training pipeline. Stage 1 (spatial grounding pre-training): the VLM is trained on large-scale multisource multimodal spatial grounding data to learn embodiment-agnostic spatial priors. Stage 2 (spatially guided action post-training): the VLM Planner, functioning as a slow but reliable System 2 reasoner, generates latent planning tokens via spatial prompting as the condition to the action expert (instantiated as a DiT Actor) to execute as a fast System 1 controller.
• Figure 3: Overview of the pre-training data for the vision-language model. The data comprises two main parts: general VQA data to maintain the model's general multimodal capabilities, and spatial VQA data focusing on robotic-related grounding and spatial perception in a VQA format.
• Figure 4: Simulation data synthesis pipeline. The pipeline generates diverse robotic manipulation data from a large asset library, converts intermediate representations into VQA data, and separates physics from rendering to reduce wasted failures and improve efficiency.
• Figure 5: Ablation study on the effect of auxiliary spatial prompting for co-training robot manipulation with spatial grounding. From left to right: (a) spatial grounding performance (IoU@0.5 on RefCOCO-g); (b) manipulation performance (SimplerEnv-WidowX SR); (c) shows the gradient similarity of the spatial grounding and manipulation objectives.