Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generation

Ruihang Xu, Dewei Zhou, Fan Ma, Yi Yang

TL;DR

ContextGen tackles identity-consistent multi-instance generation by unifying layout guidance and contextual references within a diffusion-transformer framework. It introduces Contextual Layout Anchoring (CLA) for robust spatial anchoring and Identity Consistency Attention (ICA) for preserving instance-specific details, supported by a new IMIG-100K dataset. The approach demonstrates state-of-the-art performance across layout control, identity fidelity, and visual quality on several benchmarks, and is further refined through position indexing and Direct Preference Optimization. Together, these contributions offer a scalable, data-efficient path to high-fidelity, identity-preserving multi-subject image synthesis in complex scenes.

Abstract

Multi-instance image generation (MIG) remains a significant challenge for modern diffusion models due to key limitations in achieving precise control over object layout and preserving the identity of multiple distinct subjects. To address these limitations, we introduce ContextGen, a novel Diffusion Transformer framework for multi-instance generation that is guided by both layout and reference images. Our approach integrates two key technical contributions: a Contextual Layout Anchoring (CLA) mechanism that incorporates the composite layout image into the generation context to robustly anchor the objects in their desired positions, and Identity Consistency Attention (ICA), an innovative attention mechanism that leverages contextual reference images to ensure the identity consistency of multiple instances. Recognizing the lack of large-scale, hierarchically-structured datasets for this task, we introduce IMIG-100K, the first dataset with detailed layout and identity annotations. Extensive experiments demonstrate that ContextGen sets a new state-of-the-art, outperforming existing methods in control precision, identity fidelity, and overall visual quality.

ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generation

TL;DR

ContextGen tackles identity-consistent multi-instance generation by unifying layout guidance and contextual references within a diffusion-transformer framework. It introduces Contextual Layout Anchoring (CLA) for robust spatial anchoring and Identity Consistency Attention (ICA) for preserving instance-specific details, supported by a new IMIG-100K dataset. The approach demonstrates state-of-the-art performance across layout control, identity fidelity, and visual quality on several benchmarks, and is further refined through position indexing and Direct Preference Optimization. Together, these contributions offer a scalable, data-efficient path to high-fidelity, identity-preserving multi-subject image synthesis in complex scenes.

Abstract

Multi-instance image generation (MIG) remains a significant challenge for modern diffusion models due to key limitations in achieving precise control over object layout and preserving the identity of multiple distinct subjects. To address these limitations, we introduce ContextGen, a novel Diffusion Transformer framework for multi-instance generation that is guided by both layout and reference images. Our approach integrates two key technical contributions: a Contextual Layout Anchoring (CLA) mechanism that incorporates the composite layout image into the generation context to robustly anchor the objects in their desired positions, and Identity Consistency Attention (ICA), an innovative attention mechanism that leverages contextual reference images to ensure the identity consistency of multiple instances. Recognizing the lack of large-scale, hierarchically-structured datasets for this task, we introduce IMIG-100K, the first dataset with detailed layout and identity annotations. Extensive experiments demonstrate that ContextGen sets a new state-of-the-art, outperforming existing methods in control precision, identity fidelity, and overall visual quality.

Paper Structure

This paper contains 55 sections, 5 equations, 18 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Diffusion Models
  4. Instance-level Controllable Image Generation
  5. Method
  6. Preliminaries
  7. Multimodal Diffusion Transformers (MM-DiT)
  8. Position Indexing and Attention Mask in MM-Attention
  9. Contextual Attention with Layout Anchoring and Identity Preservation
  10. Contextual Conditioning with Layout and Reference Images
  11. Contextual Layout Anchoring (CLA)
  12. Identity Consistency Attention (ICA)
  13. Instance-Wise Position Indexing
  14. IMIG-100K: An Image-Guided Multi-Instance-Generation Dataset
  15. Dataset Structure and Key Features
  16. ...and 40 more sections

Figures (18)

  • Figure 1: Representative showcases of our work. Upper panel: Our multi-subject-driven generation results versus existing open-source SOTA (OmniGen2) and proprietary models (Nano Banana, GPT-4o). Lower panel: Our layout-to-image generation examples using different layouts.
  • Figure 2: Overview of ContextGen. Left (Setup Stage): Options to composite the Layout Image. Middle (Model Core): Central generation architecture using FLUX DiT-Blocks. Right (Attention Mechanisms): Details of MM-Attention components (Position Indexing, CLA and ICA).
  • Figure 3: Image Samples of IMIG-100K Dataset.
  • Figure 4: Qualitative results on LAMICBench++.
  • Figure 5: Qualitative results on COCO-MIG. We use red dashed box to indicate the missing, merged, dislocated and incorrectly attributed instances.
  • ...and 13 more figures