SpatialLLM: From Multi-modality Data to Urban Spatial Intelligence

TL;DR

SpatialLLM introduces a training-free framework for urban spatial intelligence by converting raw multi-modality urban data into Structured Scene Descriptions (SSD) using a Multi-modality Data Joint Description (MDJD) module. SSDs are then used as prompts to pretrained LLMs to perform both basic spatial perception tasks and zero-shot advanced analyses in urban planning, ecology, and traffic management. The approach highlights that multi-field knowledge, sufficient context length, and robust reasoning significantly influence LLM spatial performance, and demonstrates notable gains over baselines. By providing a scalable, unified text-based paradigm for complex outdoor scenes, SpatialLLM offers a practical pathway to enhance urban analysis and management without specialized geospatial tools or domain-specific fine-tuning.

Abstract

We propose SpatialLLM, a novel approach advancing spatial intelligence tasks in complex urban scenes. Unlike previous methods requiring geographic analysis tools or domain expertise, SpatialLLM is a unified language model directly addressing various spatial intelligence tasks without any training, fine-tuning, or expert intervention. The core of SpatialLLM lies in constructing detailed and structured scene descriptions from raw spatial data to prompt pre-trained LLMs for scene-based analysis. Extensive experiments show that, with our designs, pretrained LLMs can accurately perceive spatial distribution information and enable zero-shot execution of advanced spatial intelligence tasks, including urban planning, ecological analysis, traffic management, etc. We argue that multi-field knowledge, context length, and reasoning ability are key factors influencing LLM performances in urban analysis. We hope that SpatialLLM will provide a novel viable perspective for urban intelligent analysis and management. The code and dataset are available at https://github.com/WHU-USI3DV/SpatialLLM.

Figures (8)

• Figure 1: We present SpatialLLM, an innovative framework that enables spatial intelligence by transforming multi-modality data into structured text. SpatialLLM first constructs comprehensive scene descriptions from raw spatial data, capturing complex object attributes and spatial relationships within a large outdoor scene. Then, by feeding these structured descriptions into pretrained LLMs, SpatialLLM can perform advanced spatial analysis including site selection, ecological analysis, etc.
• Figure 2: The overview of SpatialLLM. SpatialLLM conducts advanced spatial intelligence tasks with raw urban data inputs.
• Figure 3: Captioning polygon-type objects using multi-view images.
• Figure 4: The overview of the raw data of two evaluation scenes.
• Figure 5: Comparison of spatial reasoning capabilities between the DeepSeek-V3(red) and DeepSeek-R1(green) models. (a) shows a distance perception task where V3 uses simplified Euclidean calculations yielding incorrect results (402.4m), while R1 applies professional Haversine formula with latitude adjustments, correctly calculating 383m. (b) shows that V3 mistakenly identifies the result without position comparison, while R1 correctly determines "northwest" through systematic longitude-latitude analysis.