Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications

TL;DR

Vision-Language-Action models aim to unify perception, language, and motor control to enable generalist robots that can adapt to new tasks and embodiments with minimal task-specific data. The survey traces architectural evolution from CNN-based to transformer- and diffusion-based designs, including hierarchical policies and latent-action learning, and emphasizes full-stack coverage of data, training, datasets, and hardware. Key takeaways are the central role of large-scale foundation models, cross-embodiment and latent-action representations, and the integration of world models and affordance reasoning to support long-horizon planning and safer real-world deployment. The work provides pragmatic guidance for practitioners and highlights future directions in data modalities, reasoning, continual learning, RL, safety, evaluation, and real-world applications.

Abstract

Amid growing efforts to leverage advances in large language models (LLMs) and vision-language models (VLMs) for robotics, Vision-Language-Action (VLA) models have recently gained significant attention. By unifying vision, language, and action data at scale, which have traditionally been studied separately, VLA models aim to learn policies that generalise across diverse tasks, objects, embodiments, and environments. This generalisation capability is expected to enable robots to solve novel downstream tasks with minimal or no additional task-specific data, facilitating more flexible and scalable real-world deployment. Unlike previous surveys that focus narrowly on action representations or high-level model architectures, this work offers a comprehensive, full-stack review, integrating both software and hardware components of VLA systems. In particular, this paper provides a systematic review of VLAs, covering their strategy and architectural transition, architectures and building blocks, modality-specific processing techniques, and learning paradigms. In addition, to support the deployment of VLAs in real-world robotic applications, we also review commonly used robot platforms, data collection strategies, publicly available datasets, data augmentation methods, and evaluation benchmarks. Throughout this comprehensive survey, this paper aims to offer practical guidance for the robotics community in applying VLAs to real-world robotic systems. All references categorized by training approach, evaluation method, modality, and dataset are available in the table on our project website: https://vla-survey.github.io .

Figures (8)

• Figure 1: Structure of this survey. Section \ref{['sec:challenges']} outlines the key challenges in developing Vision-Language-Action (VLA) models. Section \ref{['sec:vla_strategy']} and Section \ref{['sec:architectures_and_building_blocks']} review the evolution of VLA strategies, architectures, and modality-specific design choices. Section \ref{['sec:training']} categorizes training strategies and practical implementation considerations. Section \ref{['sec:datasets']} discusses the data collection methologies, publicly available dataset, and data augmentation. Section \ref{['sec:real_world']} discusses the types of robots used, evaluation benchmarks, and the applications of VLA models in real-world robot systems. Guidance for practitioners is presented in Section \ref{['sec:recommendation']}, based on the findings of the systematic review.
• Figure 2: Timeline of major Vision–Language–Action (VLA) models. This figure summarizes the historical progression of representative VLA models shown in Section \ref{['sec:vla_strategy']}: from early CNN-based models (e.g., CLIPortshridhar2021cliport), to real-world scalable policies leveraging pre-trained VLM backbones (e.g., RT-1, RT-2, RT-X, OpenVLAbrohan2023rt1zitkovich2023rt2oneill2024openxembodimentkim2024openvla), followed by models integrating diffusion and flow matching techniques (e.g., Octo, RDT-1B, $\pi_0$octoteam2024octoliu2025rdtPi-0), and more recent approaches focusing on latent action inference and hierarchical control (e.g., LAPA, $\pi_{0.5}$, GR00T N1ye2025lapaPi-0.5Gr00t-N1).
• Figure 3: Structure of Section \ref{['sec:architectures_and_building_blocks']} and Section \ref{['sec:training']}. The figure summarizes key components of VLA models. The center illustrates core architectural types, including sensorimotor models, world models, and affordance-based models. The left side depicts the input and output modalities—vision, language, action, and other auxiliary modalities. The right side presents training strategies, including supervised learning, self-supervised learning, and reinforcement learning, along with practical implementation considerations.
• Figure 4: Architecture of sensorimotor models for VLA. This figure categorizes seven representative architectures used in recent VLA research. (1) Transformer + Discrete Action Token: A standard transformer processes tokenized inputs to predict discrete actions. (2) Transformer + Diffusion Action Head: A diffusion model is appended to the transformer for generating smooth, continuous actions. (3) Diffusion Transformer: The diffusion process is integrated directly within the transformer architecture. (4) VLM + Discrete Action Token: Vision-language models (VLMs) replace transformers to leverage pre-trained knowledge while predicting discrete actions. (5) VLM + Diffusion Action Head: Combines VLMs with diffusion heads for continuous control. (6) VLM + Flow Matching Action Head: Substitutes diffusion with flow matching to enhance real-time control. (7) VLM + Diffusion Transformer: Employs a VLM as a backbone and a diffusion transformer as a low-level policy for end-to-end continuous action generation.
• Figure 5: Design patterns for incorporating world models in VLA. (1) Using world models in conjunction with inverse dynamics models to generate actions. (2) Leveraging world models to learn latent action representations, particularly from human videos; the resulting latent tokens are then used for VLA training to incorporate human video datasets. (3) Generating future observations in addition to actions, enabling predictive planning and multimodal reasoning.

Theorems & Definitions (1)

• Definition 1.1: Vision-Language-Action (VLA) Model