Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation

Mingcheng Li, Xiaolu Hou, Ziyang Liu, Dingkang Yang, Ziyun Qian, Jiawei Chen, Jinjie Wei, Yue Jiang, Qingyao Xu, Lihua Zhang

TL;DR

This work addresses the challenge of generating high-fidelity images from complex text prompts by introducing MCCD, a training-free framework that combines a Multi-agent Collaboration-based Scene Parsing (MCP) with Hierarchical Compositional Diffusion (HCD). MCP decomposes complex prompts into objects, relations, backgrounds, and aesthetics through a coordinated team of specialized agents, guided by a conductor and an evaluator that implement forward reasoning and backward feedback. HCD then performs progressive, region-aware diffusion with Gaussian masks, regional enhancement, and latent-space smoothing to faithfully render overlapping objects and intricate relations. The approach yields substantial, training-free improvements over baseline diffusion models on complex-scene prompts, demonstrated by quantitative gains on T2I-CompBench and qualitative analyses, highlighting its potential for scalable, controllable T2I generation. The work also discusses broader societal implications and outlines future directions to optimize inference efficiency for large-scale prompts.

Abstract

Diffusion models have shown excellent performance in text-to-image generation. Nevertheless, existing methods often suffer from performance bottlenecks when handling complex prompts that involve multiple objects, characteristics, and relations. Therefore, we propose a Multi-agent Collaboration-based Compositional Diffusion (MCCD) for text-to-image generation for complex scenes. Specifically, we design a multi-agent collaboration-based scene parsing module that generates an agent system comprising multiple agents with distinct tasks, utilizing MLLMs to extract various scene elements effectively. In addition, Hierarchical Compositional diffusion utilizes a Gaussian mask and filtering to refine bounding box regions and enhance objects through region enhancement, resulting in the accurate and high-fidelity generation of complex scenes. Comprehensive experiments demonstrate that our MCCD significantly improves the performance of the baseline models in a training-free manner, providing a substantial advantage in complex scene generation.

MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation

TL;DR

This work addresses the challenge of generating high-fidelity images from complex text prompts by introducing MCCD, a training-free framework that combines a Multi-agent Collaboration-based Scene Parsing (MCP) with Hierarchical Compositional Diffusion (HCD). MCP decomposes complex prompts into objects, relations, backgrounds, and aesthetics through a coordinated team of specialized agents, guided by a conductor and an evaluator that implement forward reasoning and backward feedback. HCD then performs progressive, region-aware diffusion with Gaussian masks, regional enhancement, and latent-space smoothing to faithfully render overlapping objects and intricate relations. The approach yields substantial, training-free improvements over baseline diffusion models on complex-scene prompts, demonstrated by quantitative gains on T2I-CompBench and qualitative analyses, highlighting its potential for scalable, controllable T2I generation. The work also discusses broader societal implications and outlines future directions to optimize inference efficiency for large-scale prompts.

Abstract

Diffusion models have shown excellent performance in text-to-image generation. Nevertheless, existing methods often suffer from performance bottlenecks when handling complex prompts that involve multiple objects, characteristics, and relations. Therefore, we propose a Multi-agent Collaboration-based Compositional Diffusion (MCCD) for text-to-image generation for complex scenes. Specifically, we design a multi-agent collaboration-based scene parsing module that generates an agent system comprising multiple agents with distinct tasks, utilizing MLLMs to extract various scene elements effectively. In addition, Hierarchical Compositional diffusion utilizes a Gaussian mask and filtering to refine bounding box regions and enhance objects through region enhancement, resulting in the accurate and high-fidelity generation of complex scenes. Comprehensive experiments demonstrate that our MCCD significantly improves the performance of the baseline models in a training-free manner, providing a substantial advantage in complex scene generation.
Paper Structure (23 sections, 18 equations, 8 figures, 9 tables)

This paper contains 23 sections, 18 equations, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text-to-Image Generation
  4. Compositional Diffusion Generation
  5. Methodology
  6. Overall Framework
  7. Multi-agent Collaboration based Scene Parsing
  8. Hierarchical Compositional Diffusion
  9. Experiments
  10. Datasets and Evaluation Metrics
  11. Implementation Details
  12. Main Results
  13. Ablation Study
  14. Conclusion
  15. The Prompt design of MCP
  16. ...and 8 more sections

Figures (8)

  • Figure 1: The overall framework of the proposed MCCD. MCCD consists of two core components: Multi-agent Collaboration-based scene Parsing (MCP) module and Hierarchical Compositional Diffusion (HCD) module. In MCP, the blue and green arrows indicate forward CoT reasoning and backward feedback processes, respectively
  • Figure 2: Qualitative results of MCCD improving diffusion models. MCCD enhances the attribute binding and spatial relationships of the base diffusion models. The generated results have reasonable backgrounds and detailed textures with great aesthetics and realism.
  • Figure 3: Ablation results of MCCD. The poor results after removing the critical components prove that each component is crucial.
  • Figure 4: Additional qualitative results. MCCD can handle complex text prompts with multiple objects and attribute binding relationships effectively, generating reasonable bounding box layouts and producing aesthetically pleasing images with high realism.
  • Figure 5: A case to illustrate the workflow of MCP.
  • ...and 3 more figures