SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition

TL;DR

The paper tackles the inadequate coverage of spatial cognition in current multimodal LLM benchmarks by introducing a cognitively grounded five-level framework (observation to planning) and SpatialBench, a large-scale first-person video dataset with 15 spatial-reasoning tasks aligned to these levels. It proposes an adaptive, high-level overall score to compare models across hierarchical levels and validates the framework with extensive experiments on both proprietary and open-source MLLMs, plus human benchmarks. Findings show models excel at perceptual grounding but struggle with symbolic abstraction, causality, and planning, while humans consistently outperform AI on higher-level tasks. This work establishes a principled, ability-oriented methodology for evaluating hierarchical spatial cognition in MLLMs and provides a foundation for developing spatially intelligent systems.

Abstract

Spatial cognition is fundamental to real-world multimodal intelligence, allowing models to effectively interact with the physical environment. While multimodal large language models (MLLMs) have made significant strides, existing benchmarks often oversimplify spatial cognition, reducing it to a single-dimensional metric, which fails to capture the hierarchical structure and interdependence of spatial abilities. To address this gap, we propose a hierarchical spatial cognition framework that decomposes spatial intelligence into five progressively complex levels from basic observation to high-level planning. Building upon this taxonomy, we construct SpatialBench, a large-scale, fine-grained benchmark covering 15 tasks aligned with these cognitive levels. To provide a unified evaluation across heterogeneous tasks, we further introduce a high-level capability-oriented metric that reliably assesses a model's overall spatial reasoning ability. Extensive experiments over massive MLLMs reveal distinct performance stratification across cognitive levels: models exhibit strong perceptual grounding yet remain limited in symbolic reasoning, causal inference, and planning. Additional human tests demonstrate that humans perform selective, goal-directed abstraction, while MLLMs tend to over-attend to surface details without coherent spatial intent. Our work establishes the first systematic framework for measuring hierarchical spatial cognition in MLLMs, laying the foundation for future spatially intelligent systems.

Figures (9)

• Figure 1: In a parking lot scenario, the vehicle must understand the relationships of surrounding objects, reason about possible events, and plan the optimal route to reach the exit.
• Figure 2: The proposed hierarchical spatial cognitive taxonomy.
• Figure 3: An overview of SpatialBench.
• Figure 4: The statistics of SpatialBench.
• Figure 5: Comparison of models across spatial cognitive levels.