Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation

Daoan Zhang, Che Jiang, Ruoshi Xu, Biaoxiang Chen, Zijian Jin, Yutian Lu, Jianguo Zhang, Liang Yong, Jiebo Luo, Shengda Luo

TL;DR

WorldGenBench addresses the gap in evaluating T2I systems' ability to ground images in world knowledge and perform implicit reasoning. It introduces a Knowledge Checklist Score to quantify alignment with explicit knowledge-based expectations, using LLM-generated prompts across humanities and nature and GPT-4o-based evaluation. Empirical results across 22 models show diffusion-based open-source methods outperform others in general, while GPT-4o exhibits the strongest cross-domain reasoning and knowledge integration, underscoring the gap between pattern matching and true understanding. The benchmark provides a scalable, interpretable framework to steer future T2I research toward deeper knowledge grounding and inference, reducing hallucinations in real-world applications.

Abstract

Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models still struggle with prompts that require rich world knowledge and implicit reasoning: both of which are critical for producing semantically accurate, coherent, and contextually appropriate images in real-world scenarios. To address this gap, we introduce \textbf{WorldGenBench}, a benchmark designed to systematically evaluate T2I models' world knowledge grounding and implicit inferential capabilities, covering both the humanities and nature domains. We propose the \textbf{Knowledge Checklist Score}, a structured metric that measures how well generated images satisfy key semantic expectations. Experiments across 21 state-of-the-art models reveal that while diffusion models lead among open-source methods, proprietary auto-regressive models like GPT-4o exhibit significantly stronger reasoning and knowledge integration. Our findings highlight the need for deeper understanding and inference capabilities in next-generation T2I systems. Project Page: \href{https://dwanzhang-ai.github.io/WorldGenBench/}{https://dwanzhang-ai.github.io/WorldGenBench/}

WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation

TL;DR

WorldGenBench addresses the gap in evaluating T2I systems' ability to ground images in world knowledge and perform implicit reasoning. It introduces a Knowledge Checklist Score to quantify alignment with explicit knowledge-based expectations, using LLM-generated prompts across humanities and nature and GPT-4o-based evaluation. Empirical results across 22 models show diffusion-based open-source methods outperform others in general, while GPT-4o exhibits the strongest cross-domain reasoning and knowledge integration, underscoring the gap between pattern matching and true understanding. The benchmark provides a scalable, interpretable framework to steer future T2I research toward deeper knowledge grounding and inference, reducing hallucinations in real-world applications.

Abstract

Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models still struggle with prompts that require rich world knowledge and implicit reasoning: both of which are critical for producing semantically accurate, coherent, and contextually appropriate images in real-world scenarios. To address this gap, we introduce \textbf{WorldGenBench}, a benchmark designed to systematically evaluate T2I models' world knowledge grounding and implicit inferential capabilities, covering both the humanities and nature domains. We propose the \textbf{Knowledge Checklist Score}, a structured metric that measures how well generated images satisfy key semantic expectations. Experiments across 21 state-of-the-art models reveal that while diffusion models lead among open-source methods, proprietary auto-regressive models like GPT-4o exhibit significantly stronger reasoning and knowledge integration. Our findings highlight the need for deeper understanding and inference capabilities in next-generation T2I systems. Project Page: \href{https://dwanzhang-ai.github.io/WorldGenBench/}{https://dwanzhang-ai.github.io/WorldGenBench/}
Paper Structure (13 sections, 1 equation, 7 figures, 4 tables)

This paper contains 13 sections, 1 equation, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Evaluation of Text-to-Image Models
  4. WorldGenBench
  5. WorldGenBench Construction
  6. Evaluation: Knowledge Checklist Score
  7. Results
  8. Conclusion
  9. Abbreviations
  10. Evaluation Prompt for Knowledge Checklist Score,
  11. Visual Cases
  12. Checklist for Visual Case 1
  13. Checklist for Visual Case 2

Figures (7)

  • Figure 1: The presence of world knowledge and the emergence of implicit reasoning capabilities are fundamental to building a high-quality text-to-image model.
  • Figure 2: Detailed static information of WorldGenBench.
  • Figure 3: The Construction and Evaluation Pipeline of WorldGenBench.
  • Figure 4: Examples from WorldGenBench: Input Prompt and Corresponding Checklist. Left: Humanities; Right: Nature.
  • Figure 5: Visual Case 1: An example case from the humanities domain, including the prompt and the results from GPT-4o, HiDreaml1-Full, and Show-o. Checklist is in \ref{['111']}
  • ...and 2 more figures