CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments

TL;DR

CityCube introduces a large-scale benchmark for cross-view spatial reasoning in urban environments, combining real-world and simulated urban imagery to create 5,022 MCQ QA pairs across 59 tasks, five cognitive categories, and three spatial relation expressions. It uses four observation dynamics and three camera perspectives to test VLMs on perception-grounded, perspective-taking, and mental reconstruction tasks, with a diverse evaluation of 33 VLMs and a rigorous data-collection-and-generation pipeline including role-based QA generation and human verification. The results reveal a substantial AI-human gap and indicate that existing reasoning-oriented models do not reliably improve cross-view spatial tasks; fine-tuning with CityCube data (via LoRA) yields gains, and task correlations reveal both shared and distinct spatial capabilities. CityCube demonstrates the limits of current VLM frameworks for urban CvSI, provides insights into failure modes, and offers a diagnostic platform to drive spatially grounded learning and the design of next-generation urban spatial intelligence systems.

Abstract

Cross-view spatial reasoning is essential for embodied AI, underpinning spatial understanding, mental simulation and planning in complex environments. Existing benchmarks primarily emphasize indoor or street settings, overlooking the unique challenges of open-ended urban spaces characterized by rich semantics, complex geometries, and view variations. To address this, we introduce CityCube, a systematic benchmark designed to probe cross-view reasoning capabilities of current VLMs in urban settings. CityCube integrates four viewpoint dynamics to mimic camera movements and spans a wide spectrum of perspectives from multiple platforms, e.g., vehicles, drones and satellites. For a comprehensive assessment, it features 5,022 meticulously annotated multi-view QA pairs categorized into five cognitive dimensions and three spatial relation expressions. A comprehensive evaluation of 33 VLMs reveals a significant performance disparity with humans: even large-scale models struggle to exceed 54.1% accuracy, remaining 34.2% below human performance. By contrast, small-scale fine-tuned VLMs achieve over 60.0% accuracy, highlighting the necessity of our benchmark. Further analyses indicate the task correlations and fundamental cognitive disparity between VLMs and human-like reasoning.

Figures (21)

• Figure 1: Illustration of the CityCube benchmark. Left: An illustration of an embodied orbiting observation, where an agent captures multi-view images by circling a focal object (highlighted in red). Middle: Examples of multi-choice QA designed to evaluate five dimensions of CvSI. Right: Task distributions on CityCube Benchmark.
• Figure 2: The systematic evaluation protocol of the CityCube benchmark. Upper Left: Dim 1 evaluates observation with four representative behavior; Upper Right: Dim 2 tests model across various camera orientations; Bottom Left: Dim 3 categorizes 59 tasks into 5 fundamental categories; and Bottom Right: Dim 4 labels the QA pairs with spatial reference frames.
• Figure 3: Task correlation matrix. Each axis corresponds to the 59 tasks after classification, and color intensity indicates the strength of correlations.
• Figure 4: Human vs AI task correlation. The scatter plots illustrate the performance correlation across 59 tasks, where each individual point represents a specific task.
• Figure 5: Illustration of the CityCube-Bench construction pipeline. Images are collected from diverse real-world datasets and urban simulators; relevant image sets are carefully selected; complex QA tasks and detailed reasoning processes are annotated through human–AI collaboration; and all data undergo rigorous quality control.