Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models

Zengbin Wang, Xuecai Hu, Yong Wang, Feng Xiong, Man Zhang, Xiangxiang Chu

TL;DR

SpatialGenEval addresses a critical gap in evaluating text-to-image models by focusing on spatial intelligence—how models perceive, reason about, and interact with spatial relationships in real-world scenes. The authors design long, information-dense prompts spanning 10 spatial sub-domains across 25 scenes and pair each prompt with omni-dimensional multiple-choice questions, enabling fine-grained diagnostics. Evaluations across 23 state-of-the-art models reveal a universal bottleneck in higher-order spatial reasoning, with foundation-generation tasks outperforming complex perception, reasoning, and interaction tasks; text encoder quality and model scale emerge as key determinants of spatial performance. To translate evaluation into improvement, the SpatialT2I dataset is created for supervised fine-tuning, and fine-tuning on models like Stable Diffusion-XL, UniWorld-V1, and OmniGen2 yields consistent gains, validating a data-centric approach for enhancing spatial intelligence in T2I systems. The work also provides robust evaluation protocols using multiple LLM judges and human alignment, supporting reproducibility and practical impact for developers and researchers seeking more spatially capable generative models.

Abstract

Text-to-image (T2I) models have achieved remarkable success in generating high-fidelity images, but they often fail in handling complex spatial relationships, e.g., spatial perception, reasoning, or interaction. These critical aspects are largely overlooked by current benchmarks due to their short or information-sparse prompt design. In this paper, we introduce SpatialGenEval, a new benchmark designed to systematically evaluate the spatial intelligence of T2I models, covering two key aspects: (1) SpatialGenEval involves 1,230 long, information-dense prompts across 25 real-world scenes. Each prompt integrates 10 spatial sub-domains and corresponding 10 multi-choice question-answer pairs, ranging from object position and layout to occlusion and causality. Our extensive evaluation of 21 state-of-the-art models reveals that higher-order spatial reasoning remains a primary bottleneck. (2) To demonstrate that the utility of our information-dense design goes beyond simple evaluation, we also construct the SpatialT2I dataset. It contains 15,400 text-image pairs with rewritten prompts to ensure image consistency while preserving information density. Fine-tuned results on current foundation models (i.e., Stable Diffusion-XL, Uniworld-V1, OmniGen2) yield consistent performance gains (+4.2%, +5.7%, +4.4%) and more realistic effects in spatial relations, highlighting a data-centric paradigm to achieve spatial intelligence in T2I models.

Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models

TL;DR

SpatialGenEval addresses a critical gap in evaluating text-to-image models by focusing on spatial intelligence—how models perceive, reason about, and interact with spatial relationships in real-world scenes. The authors design long, information-dense prompts spanning 10 spatial sub-domains across 25 scenes and pair each prompt with omni-dimensional multiple-choice questions, enabling fine-grained diagnostics. Evaluations across 23 state-of-the-art models reveal a universal bottleneck in higher-order spatial reasoning, with foundation-generation tasks outperforming complex perception, reasoning, and interaction tasks; text encoder quality and model scale emerge as key determinants of spatial performance. To translate evaluation into improvement, the SpatialT2I dataset is created for supervised fine-tuning, and fine-tuning on models like Stable Diffusion-XL, UniWorld-V1, and OmniGen2 yields consistent gains, validating a data-centric approach for enhancing spatial intelligence in T2I systems. The work also provides robust evaluation protocols using multiple LLM judges and human alignment, supporting reproducibility and practical impact for developers and researchers seeking more spatially capable generative models.

Abstract

Text-to-image (T2I) models have achieved remarkable success in generating high-fidelity images, but they often fail in handling complex spatial relationships, e.g., spatial perception, reasoning, or interaction. These critical aspects are largely overlooked by current benchmarks due to their short or information-sparse prompt design. In this paper, we introduce SpatialGenEval, a new benchmark designed to systematically evaluate the spatial intelligence of T2I models, covering two key aspects: (1) SpatialGenEval involves 1,230 long, information-dense prompts across 25 real-world scenes. Each prompt integrates 10 spatial sub-domains and corresponding 10 multi-choice question-answer pairs, ranging from object position and layout to occlusion and causality. Our extensive evaluation of 21 state-of-the-art models reveals that higher-order spatial reasoning remains a primary bottleneck. (2) To demonstrate that the utility of our information-dense design goes beyond simple evaluation, we also construct the SpatialT2I dataset. It contains 15,400 text-image pairs with rewritten prompts to ensure image consistency while preserving information density. Fine-tuned results on current foundation models (i.e., Stable Diffusion-XL, Uniworld-V1, OmniGen2) yield consistent performance gains (+4.2%, +5.7%, +4.4%) and more realistic effects in spatial relations, highlighting a data-centric paradigm to achieve spatial intelligence in T2I models.
Paper Structure (29 sections, 11 figures, 12 tables)

This paper contains 29 sections, 11 figures, 12 tables.

Table of Contents

  1. Introduction
  2. SpatialGenEval Benchmark
  3. Key Principles
  4. Focused Aspects
  5. Benchmark Construction
  6. Information-dense & Spatial-aware Prompt Generation
  7. Omni-dimensional QAs Generation
  8. Experimental Results
  9. Setup
  10. Main Results of SpatialGenEval Benchmark
  11. Supervised Fine-Tuning (SFT)
  12. Related Work
  13. Conclusion
  14. Appendix
  15. Benchmark Statistics and Analysis
  16. ...and 14 more sections

Figures (11)

  • Figure 1: (Top): Error cases around spatial perception, reasoning, and interaction from GPT-Image-1 hurst2024gpt4o-image, Qwen-Image wu2025qwen-image, and Bagel deng2025bagel. (Bottom): A comparison of prompt and evaluation formats across current benchmarks wei2025TIIF-Bench.
  • Figure 2: Overview of the SpatialGenEval benchmark and key results. The benchmark is structured around (a) 10 spatial sub-domains and (b) 25 real-world scenes. (c) The evaluation of 23 SOTA T2I models shows the overall performance ranking and (d) a detailed capability breakdown.
  • Figure 3: Examples of SpatialGenEval. Each image is generated from an information-dense prompt covering all 10 spatial sub-domains and evaluated with 10 corresponding multiple-choice questions.
  • Figure 4: SpatialGenEval Construction Pipeline. (a) The process begins by selecting one of 25 real-world scenes and combining it with the definitions of all 10 spatial sub-domains. (b) The MLLM sequentially synthesizes an information-dense prompt that integrates all 10 constraints, along with 10 corresponding omni-dimensional QA pairs. (c) T2I models generate an image from the prompt, which is then evaluated against the QA pairs to yield a fine-grained spatial intelligence score.
  • Figure 5: Distribution of error types across scenes (left, based on all T2I models) and some examples of T2I models (right) in our SpatialGenEval.
  • ...and 6 more figures