SpatialTree: How Spatial Abilities Branch Out in MLLMs

TL;DR

SpatialTree introduces a four-level, capability-centric taxonomy of spatial abilities for multimodal LLMs, grouping skills into L1 Perception, L2 Mental Mapping, L3 Mental Simulation, and L4 Agentic Competence. It presents SpatialTree-Bench, a hierarchical benchmark and SpatialEngine-based data engine to evaluate 27 sub-abilities across L1–L4, revealing that L1 skills are largely independent while higher levels exhibit strong interdependencies and transfer dynamics. Targeted Supervised Fine-Tuning demonstrates cross-level transfer from low-level perception to high-level reasoning and action, and highlights synergy when multiple foundational abilities are trained together. The work also shows that naive reinforcement learning is not uniformly beneficial; a simple auto-think strategy with hierarchy-aware rewards consistently improves performance across the hierarchy, providing a practical path toward scaling spatial intelligence in MLLMs.

Abstract

Cognitive science suggests that spatial ability develops progressively-from perception to reasoning and interaction. Yet in multimodal LLMs (MLLMs), this hierarchy remains poorly understood, as most studies focus on a narrow set of tasks. We introduce SpatialTree, a cognitive-science-inspired hierarchy that organizes spatial abilities into four levels: low-level perception (L1), mental mapping (L2), simulation (L3), and agentic competence (L4). Based on this taxonomy, we construct the first capability-centric hierarchical benchmark, thoroughly evaluating mainstream MLLMs across 27 sub-abilities. The evaluation results reveal a clear structure: L1 skills are largely orthogonal, whereas higher-level skills are strongly correlated, indicating increasing interdependency. Through targeted supervised fine-tuning, we uncover a surprising transfer dynamic-negative transfer within L1, but strong cross-level transfer from low- to high-level abilities with notable synergy. Finally, we explore how to improve the entire hierarchy. We find that naive RL that encourages extensive "thinking" is unreliable: it helps complex reasoning but hurts intuitive perception. We propose a simple auto-think strategy that suppresses unnecessary deliberation, enabling RL to consistently improve performance across all levels. By building SpatialTree, we provide a proof-of-concept framework for understanding and systematically scaling spatial abilities in MLLMs.

Figures (14)

• Figure 1: SpatialTree. Inspired by cognitive science, our proposed SpatialTree organizes spatial intelligence into a four-layer hierarchy (L1-L4). Rooted in foundational multi-modal capabilities (L0), the tree progressively branches from Basic perception (L1) to agentic competence (L4).
• Figure 2: Different Emphasis across Hierarchy Levels. Taking Relation in L1, L2, L3 as an example: (a) Relation in Perception, involving basic spatial relations (e.g., inside, outside) (b) Relation in Understanding, describing the attributes and mutual relationships among different objects. (c) Relation in Causal Reasoning, leveraging visual cues and logical inference to solve more complex relational tasks.
• Figure 3: Benchmark Data Engines. Level-specific engines process data and construct QAs.
• Figure 4: Distribution of Benchmark data and Evaluation Metrics. We analyze the metric usage across 41 tasks in our benchmark. The evaluation relies primarily on multiple-choice questions (70.7%), complemented by task-specific numeric metrics (e.g., cognitive map accuracy) and LLM-as-a-Judge protocols.
• Figure 5: Inter-Capability Dependencies via Pearson Correlation. (A) Correlation matrix among higher-level capabilities (L3 and L4); (B) Correlation matrix among foundational L1 capabilities; (C) Salient low-level abilities influencing higher-level tasks.