Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding

TL;DR

The paper tackles the有限 spatial understanding of Vision-Language-Action (VLA) systems by introducing Evo-0, a plug-and-play architecture that implicitly encodes 3D geometry through a Visual Geometry Grounded Transformer (VGGT) spatial encoder. By cross-attentively fusing VGGT 3D tokens with standard 2D visual tokens and keeping the Vision-Language Model frozen (with LoRA adapters), Evo-0 delivers depth-aware representations from RGB images alone and improves policy learning. Extensive simulations on RLBench tasks, real-world experiments, and robustness tests demonstrate consistent, substantial gains over strong baselines, including better precision in tight spatial tasks and resilience to disturbances. The approach offers a practical, sensor-light path to richer spatial grounding in robotic manipulation with broad applicability in real-world settings.

Abstract

Vision-Language-Action (VLA) models have emerged as a promising framework for enabling generalist robots capable of perceiving, reasoning, and acting in the real world. These models usually build upon pretrained Vision-Language Models (VLMs), which excel at semantic understanding due to large-scale image and text pretraining. However, existing VLMs typically lack precise spatial understanding capabilities, as they are primarily tuned on 2D image-text pairs without 3D supervision. To address this limitation, recent approaches have incorporated explicit 3D inputs such as point clouds or depth maps, but this necessitates additional depth sensors or pre-trained depth estimation models, which may yield defective results. In contrast, our work introduces a plug-and-play module that implicitly incorporates 3D geometry features into VLA models by leveraging an off-the-shelf visual geometry foundation model. This integration provides the model with depth-aware visual representations, improving its ability to understand the geometric structure of the scene and the spatial relationships among objects from RGB images alone. We evaluate our method on a set of spatially challenging tasks in both simulation and the real world. Extensive evaluations show that our method significantly improves the performance of state-of-the-art VLA models across diverse scenarios.

Figures (6)

• Figure 1: Architecture of Evo-0. The input RGB images are initially processed by a 2D image encoder and a VGGT spatial encoder. The extracted features are subsequently fused through a fusion layer to form a spatially enriched visual representation. This representation is further propagated through a vision language model and an action module to produce the robot actions. This spatially-enhanced perception pipeline leads to strong performance across the simulation, real-world, and disturbance tasks illustrated on the right side of the figure.
• Figure 2: Simulation Experiments. We evaluate all models on five RLBench simulation tasks requiring precise spatial manipulation, using a multi-task training setup. Success rates are shown per task and averaged. The success criteria are defined according to the official RLBench specifications.
• Figure 3: Illustration of the task setup. Five real-world evaluation tasks are used in our experiments: centering a cylinder on a target, peg-in-hole insertion, middle bottle grasping, can pick-and-place, and transparent object pick-and-place.
• Figure 4: Hyperparameter Experiments. We visualize the impact of training steps and horizon on success rates. The results highlight the training efficiency of our method and its robustness to horizon variation.
• Figure 5: Qualitative results in real-world tasks. Detailed execution sequences of the five real-world evaluation tasks are visualized, illustrating the step-by-step progression for each scenario.