InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation

TL;DR

InternVLA-A1 addresses the gap between semantic understanding and physical dynamics in robotic manipulation by unifying understanding, generation, and action through a Mixture-of-Transformers framework. It combines MLLM-based understanding with a VAE latent generator and a flow-matching action module, all coordinated via a masked self-attention scheme and a hierarchical data pyramid that blends synthetic InternData-A1 with AgiBot-World and specialized real data. The model comes in 2B and 3B scales, achieves about 13 Hz real-time inference, and is pre-trained with a large synthetic-real data mix before task-specific fine-tuning. Empirical results across twelve real-world tasks and a RoboTwin simulation benchmark show consistent superiority over strong baselines, particularly in dynamic settings, validating the approach's effectiveness in robust, dynamics-aware manipulation.

Abstract

Prevalent Vision-Language-Action (VLA) models are typically built upon Multimodal Large Language Models (MLLMs) and demonstrate exceptional proficiency in semantic understanding, but they inherently lack the capability to deduce physical world dynamics. Consequently, recent approaches have shifted toward World Models, typically formulated via video prediction; however, these methods often suffer from a lack of semantic grounding and exhibit brittleness when handling prediction errors. To synergize semantic understanding with dynamic predictive capabilities, we present InternVLA-A1. This model employs a unified Mixture-of-Transformers architecture, coordinating three experts for scene understanding, visual foresight generation, and action execution. These components interact seamlessly through a unified masked self-attention mechanism. Building upon InternVL3 and Qwen3-VL, we instantiate InternVLA-A1 at 2B and 3B parameter scales. We pre-train these models on hybrid synthetic-real datasets spanning InternData-A1 and Agibot-World, covering over 533M frames. This hybrid training strategy effectively harnesses the diversity of synthetic simulation data while minimizing the sim-to-real gap. We evaluated InternVLA-A1 across 12 real-world robotic tasks and simulation benchmark. It significantly outperforms leading models like pi0 and GR00T N1.5, achieving a 14.5\% improvement in daily tasks and a 40\%-73.3\% boost in dynamic settings, such as conveyor belt sorting.

Figures (9)

• Figure 1: InternVLA-A1 integrates understanding, generation, and action experts into a unified model, which synergizes semantic reasoning with dynamics prediction to guide action execution. By coupling this architectural design with pretraining on hybrid synthetic-real datasets, the model exhibits consistent robustness across diverse tasks, with remarkable superiority in dynamic scenarios.
• Figure 2: Framework of InternVLA-A1. The architecture comprises three experts: (1) an understanding expert that encodes scene context from image and text inputs; (2) a generation expert that predicts future visual states and task dynamics; and (3) an action expert that integrates the encoded scene context with these predictive dynamics to synthesize control commands via Flow Matching. This tripartite design enables robust manipulation across diverse scenarios.
• Figure 3: The hierarchical data pyramid.
• Figure 4: The experimental setting of general-purpose tasks.
• Figure 5: The experimental setting of specialized tasks: (a) Express Sorting task, (b) In-motion Ingredient Picking task.