Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Right Looks, Wrong Reasons: Compositional Fidelity in Text-to-Image Generation

Mayank Vatsa, Aparna Bharati, Richa Singh

TL;DR

This paper analyzes compositional fidelity in text-to-image generation across negation, counting, and spatial relations, showing a submultiplicative drop when constraints are combined. It provides a formal account of joint composition failures, traces the data-architecture mismatch and inter-primitive interference, and surveys 15 benchmarks and methods. It synthesizes approaches across data augmentation, architectural controls, and neuro-symbolic strategies, and highlights evaluation gaps and the need for principled benchmarks. The authors argue that progress will require fundamental advances in representation and reasoning, including explicit discrete reasoning components and constraint-focused training, to enable trustworthy and controllable image generation.

Abstract

The architectural blueprint of today's leading text-to-image models contains a fundamental flaw: an inability to handle logical composition. This survey investigates this breakdown across three core primitives-negation, counting, and spatial relations. Our analysis reveals a dramatic performance collapse: models that are accurate on single primitives fail precipitously when these are combined, exposing severe interference. We trace this failure to three key factors. First, training data show a near-total absence of explicit negations. Second, continuous attention architectures are fundamentally unsuitable for discrete logic. Third, evaluation metrics reward visual plausibility over constraint satisfaction. By analyzing recent benchmarks and methods, we show that current solutions and simple scaling cannot bridge this gap. Achieving genuine compositionality, we conclude, will require fundamental advances in representation and reasoning rather than incremental adjustments to existing architectures.

Right Looks, Wrong Reasons: Compositional Fidelity in Text-to-Image Generation

TL;DR

This paper analyzes compositional fidelity in text-to-image generation across negation, counting, and spatial relations, showing a submultiplicative drop when constraints are combined. It provides a formal account of joint composition failures, traces the data-architecture mismatch and inter-primitive interference, and surveys 15 benchmarks and methods. It synthesizes approaches across data augmentation, architectural controls, and neuro-symbolic strategies, and highlights evaluation gaps and the need for principled benchmarks. The authors argue that progress will require fundamental advances in representation and reasoning, including explicit discrete reasoning components and constraint-focused training, to enable trustworthy and controllable image generation.

Abstract

The architectural blueprint of today's leading text-to-image models contains a fundamental flaw: an inability to handle logical composition. This survey investigates this breakdown across three core primitives-negation, counting, and spatial relations. Our analysis reveals a dramatic performance collapse: models that are accurate on single primitives fail precipitously when these are combined, exposing severe interference. We trace this failure to three key factors. First, training data show a near-total absence of explicit negations. Second, continuous attention architectures are fundamentally unsuitable for discrete logic. Third, evaluation metrics reward visual plausibility over constraint satisfaction. By analyzing recent benchmarks and methods, we show that current solutions and simple scaling cannot bridge this gap. Achieving genuine compositionality, we conclude, will require fundamental advances in representation and reasoning rather than incremental adjustments to existing architectures.

Paper Structure

This paper contains 12 sections, 8 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Definition & Significance of Primitives in T2I Gen.
  4. Negation - The Challenge of Concept Absence:
  5. Counting - Precise Cardinality Control:
  6. Spatial Relations - Geometric & Topological Reasoning:
  7. Joint Compositionality - The Compounding Challenge:
  8. Literature Review
  9. Synthesis and Open Challenges
  10. Critical Research Gaps
  11. Conclusion
  12. Acknowledgments

Figures (1)

  • Figure 1: Examples of state-of-the-art Text-to-Image Generation models failing to correctly bind attributes and satisfy spatial relations in complex prompts.