DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning

TL;DR

DepthVLA addresses the spatial reasoning gap in Vision-Language-Action models by introducing a pretrained depth expert into a mixture-of-transformers framework. The depth expert provides explicit geometric priors, enabling end-to-end training that fuses semantic grounding with 3D geometry, while maintaining practical latency. Across real-world and simulated benchmarks, DepthVLA achieves substantial improvements in tasks requiring precise manipulation, grasping, and collision avoidance, outperforming state-of-the-art VLAs. Ablation studies show the importance of depth pretraining and depth-loss, and demonstrate that depth-prediction fosters better multimodal integration than relying solely on ground-truth depth, underscoring the value of integrated 3D priors for robust robotic control.

Abstract

Vision-Language-Action (VLA) models have recently shown impressive generalization and language-guided manipulation capabilities. However, their performance degrades on tasks requiring precise spatial reasoning due to limited spatial reasoning inherited from Vision-Language Models (VLMs). Existing VLAs rely on extensive action-data pretraining to ground VLMs in 3D space, which reduces training efficiency and is still insufficient for accurate spatial understanding. In this work, we present DepthVLA, a simple yet effective VLA architecture that explicitly incorporates spatial awareness through a pretrained depth prediction module. DepthVLA adopts a mixture-of-transformers design that unifies a VLM, a depth transformer, and an action expert with fully shared attentions, forming an end-to-end model with enhanced spatial reasoning. Extensive evaluations in both real-world and simulated environments show that DepthVLA outperforms state-of-the-art approaches, achieving 78.5% vs. 65.0% progress in real-world tasks, 94.9% vs. 93.6% in the LIBERO simulator, and 74.8% vs. 58.8% in the Simpler simulator. Our code will be made publicly available.

Figures (5)

• Figure 1: We propose DepthVLA, a vision-language-action (VLA) model that explicitly incorporates spatial reasoning through a pretrained depth expert. Quantitative comparisons across Simpler, LIBERO, and real-world benchmarks show that DepthVLA consistently outperforms baselines, particularly in tasks requiring fine-grained 3D perception.
• Figure 2: The proposed mixture-of-transformers (MoT) framework integrates three experts: a vision-language model (VLM) for semantic and language understanding, a depth expert for geometric reasoning, and an action expert for continuous control. Attention layers are shared across experts, while block-wise masking ensures pretrained modules retain their learned abilities. The action expert attends to features from both the VLM and depth expert at every layer to generate actions conditioned on language, visual, and spatial cues.
• Figure 3: Real-robot experiment platform.
• Figure 4: Performance of DepthVLA and baseline on three bimanual tasks with standard fine-tuning and few-shot adaptation. DepthVLA shows improvements in tasks requiring precise spatial reasoning and collision avoidance while maintaining comparable performance on simpler small-object manipulation.
• Figure 5: Qualitative results of DepthVLA’s predicted depth maps across real-world and simulated environments. The predicted depth provides fine-grained geometric cues that guide accurate manipulation, collision avoidance, and precise object grasping. Even in cluttered or challenging scenes, DepthVLA captures the 3D layout robustly, highlighting the effectiveness of the pretrained depth expert in providing spatial awareness.