Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes

TL;DR

This work introduces MV-RoboBench, the first benchmark explicitly designed to evaluate multi-view spatial reasoning in robotic manipulation by integrating synchronized multi-camera inputs with both spatial understanding and robotic execution tasks. By testing a wide range of models, including open-source VLMs and proprietary reasoning systems, the study reveals a substantial gap between current models and human performance, with strong improvements mainly in reasoning-enabled architectures. CoT-inspired prompts show model-dependent effects, underscoring that generic prompting cannot reliably boost multi-view robotic reasoning. A key finding is the positive correlation between spatial and robotic intelligence in capable models, but single-view spatial proficiency does not reliably transfer to multi-view embodied tasks, highlighting the need for geometry-aware, action-grounded training and evaluation. MV-RoboBench provides datasets, templates, and protocols that can catalyze progress toward truly embodied, spatially grounded vision-language systems for robotics.

Abstract

Vision-language models (VLMs) are essential to Embodied AI, enabling robots to perceive, reason, and act in complex environments. They also serve as the foundation for the recent Vision-Language-Action (VLA) models. Yet most evaluations of VLMs focus on single-view settings, leaving their ability to integrate multi-view information underexplored. At the same time, multi-camera setups are increasingly standard in robotic platforms, as they provide complementary perspectives to mitigate occlusion and depth ambiguity. Whether VLMs can effectively leverage such multi-view inputs for robotic reasoning therefore remains an open question. To bridge this gap, we introduce MV-RoboBench, a benchmark specifically designed to evaluate the multi-view spatial reasoning capabilities of VLMs in robotic manipulation. MV-RoboBench consists of 1.7k manually curated QA items across eight subtasks, divided into two primary categories: spatial understanding and robotic execution. We evaluate a diverse set of existing VLMs, including both open-source and closed-source models, along with enhanced versions incorporating CoT-inspired techniques. The results show that state-of-the-art models remain far below human performance, underscoring the substantial challenges VLMs face in multi-view robotic perception. Additionally, our analysis uncovers two key findings: (i) spatial intelligence and robotic task execution are positively correlated in multi-view robotic scenarios; and (ii) strong performance on existing general-purpose single-view spatial understanding benchmarks does not reliably translate to success in the robotic spatial tasks assessed by our benchmark. We release MV-RoboBench as an open resource to foster progress in spatially grounded VLMs and VLAs, providing not only data but also a standardized evaluation protocol for multi-view embodied reasoning.

Figures (28)

• Figure 1: Representative multi-view QA instances from the eight tasks in MV-RoboBench, with spatial tasks shown on the left and robotic tasks on the right. For clarity, only simplified versions with ground-truth answers are presented here, omitting distractors. Full examples are provided in Appendix \ref{['appendixf']}.
• Figure 2: Construction pipeline of MV-RoboBench, consisting of three stages: data collection, QA generation, and human-in-the-loop quality review.
• Figure 3: Data distribution of MV-RoboBench, showing QA counts per subtask and dataset source (AgiWorld and BridgeV2), and the overall balance between spatial and robotic domains.
• Figure 4: Leading models vs. human performance on MV-RoboBench.
• Figure 5: Spatial vs. robotic accuracy on MV-RoboBench. Models clustered near the lower-left operate close to random guessing, while reasoning-enhanced proprietary models show a clear upward trend across both axes.