SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models

TL;DR

SpatialLadder addresses the perception–reasoning gap in visual spatial reasoning for vision–language models by introducing SpatialLadder-26k, a comprehensive multimodal dataset, and a three-stage progressive training framework. The stages establish perceptual grounding through localization, cultivate spatial understanding across seven dimensions with multimodal data, and strengthen complex reasoning via reinforcement learning with verifiable rewards. The approach achieves state-of-the-art spatial reasoning on multiple benchmarks and shows strong out-of-domain generalization, validating the power of progressive spatial learning. This work provides a practical blueprint for building robust spatial intelligence in VLMs, with potential implications for embodied AI and robotics.

Abstract

Spatial reasoning remains a fundamental challenge for Vision-Language Models (VLMs), with current approaches struggling to achieve robust performance despite recent advances. We identify that this limitation stems from a critical gap: existing methods attempt to learn spatial reasoning directly without establishing the hierarchical foundations of perception and understanding. To address this challenge, we present a comprehensive methodology for building spatial intelligence progressively. We introduce SpatialLadder-26k, a multimodal dataset containing 26,610 samples spanning object localization, single image, multi-view, and video spatial reasoning tasks, constructed through a standardized pipeline that ensures systematic coverage across modalities. Building on this dataset, we design a three-stage progressive training framework that (1) establishes spatial perception through object localization, (2) develops spatial understanding through multi-dimensional spatial tasks, and (3) strengthens complex reasoning via reinforcement learning with verifiable rewards. This approach yields SpatialLadder, a 3B-parameter model that achieves state-of-the-art performance on spatial reasoning benchmarks, with 23.4% average improvement over the base model, surpassing GPT-4o by 20.8% and Gemini-2.0-Flash by 10.1%. Notably, SpatialLadder maintains strong generalization with 7.2% improvement on out-of-domain benchmarks, demonstrating that progressive training from perception to reasoning is essential for robust spatial intelligence.

Figures (12)

• Figure 1: Perception-reasoning gap in spatial reasoning. Left: Three experimental conditions with increasing perceptual hints: no hints, location hints (bounding boxes), and full hints (boxes plus directional cues). Right: While Qwen2.5-VL-3B shows progressive improvement with increasing hints, our trained model achieves superior performance with negligible reliance on external prompts.
• Figure 2: Overview of SpatialLadder-26$k$ dataset construction pipeline from raw data collection to question–answer pairs generation, with representative tasks including multi-view relative direction, single-image absolute distance, and corresponding grounding tasks.
• Figure 3: Three-stage progressive training framework of SpatialLadder. Stage 1 establishes perceptual grounding through object localization, Stage 2 develops spatial understanding across seven dimensions using multimodal tasks, and Stage 3 employs GRPO reinforcement learning with chain-of-thought generation to strengthen reasoning capabilities.
• Figure 4: Impact of progressive training stages. Left: accuracy rewards over training steps; Middle: reward standard deviation over training steps; Right: VSI-Bench performance comparison.
• Figure 5: Ablation study results.