MoGen: A Unified Collaborative Framework for Controllable Multi-Object Image Generation

TL;DR

MoGen tackles the challenge of precise, multi-object image generation by decoupling global language semantics from localized region semantics. It introduces Regional Semantic Anchor (RSA) to align phrase-level text with image regions and Adaptive Multi-modal Guidance (AMG) to flexibly fuse multi-source controls into structured intent, enabling text-driven generation with explicit layout and attribute constraints. The MoCA benchmark supports fine-grained annotation for multi-object scenes. Empirically, MoGen outperforms baselines in quantity consistency, image quality, and controllable fidelity, with ablations confirming the complementary benefits of RSA and AMG and their robust interaction. This work advances accessible, flexible, and high-fidelity controllable generation suitable for diverse resources and constraints.

Abstract

Existing multi-object image generation methods face difficulties in achieving precise alignment between localized image generation regions and their corresponding semantics based on language descriptions, frequently resulting in inconsistent object quantities and attribute aliasing. To mitigate this limitation, mainstream approaches typically rely on external control signals to explicitly constrain the spatial layout, local semantic and visual attributes of images. However, this strong dependency makes the input format rigid, rendering it incompatible with the heterogeneous resource conditions of users and diverse constraint requirements. To address these challenges, we propose MoGen, a user-friendly multi-object image generation method. First, we design a Regional Semantic Anchor (RSA) module that precisely anchors phrase units in language descriptions to their corresponding image regions during the generation process, enabling text-to-image generation that follows quantity specifications for multiple objects. Building upon this foundation, we further introduce an Adaptive Multi-modal Guidance (AMG) module, which adaptively parses and integrates various combinations of multi-source control signals to formulate corresponding structured intent. This intent subsequently guides selective constraints on scene layouts and object attributes, achieving dynamic fine-grained control. Experimental results demonstrate that MoGen significantly outperforms existing methods in generation quality, quantity consistency, and fine-grained control, while exhibiting superior accessibility and control flexibility. Code is available at: https://github.com/Tear-kitty/MoGen/tree/master.

Figures (12)

• Figure 1: High-quality samples from our MoGen. MoGen enables multi-object image generation with fine-grained control via text, optionally augmented with bounding boxes, structure references, and object references.
• Figure 2: The comparison between previous methods and our MoGen. (a) Text-to-image methods offer high accessibility but often fail to ensure quantity consistency and lack fine-grained control over scene layouts and object attributes. (b) Signal-based methods enable fine-grained control, thereby improving quantity consistency, yet their reliance on the rigid input format limits accessibility and control flexibility. (c) MoGen enables text-to-image generation that follows quantity requirements, and can further integrate various combinations of multi-source control signals. Depending on the activated signals, the model imposes corresponding fine-grained control over scene layouts and object attributes.
• Figure 3: The framework of MoGen. Regional Semantic Anchor (RSA) module decomposes input text prompts into global text semantics $\mathbf{T}_{glob}$ and phrase-level text semantics $\mathbf{T}_{phr}$. Then, Synergistic Utilization Mechanism broadcasts $\mathbf{V}_{glob}'$ (derived from $\mathbf{T}_{glob}$) across all U-Net blocks to ensure structural stability of generated images, while integrating $\mathbf{V}_{phr}'$ (derived from $\mathbf{T}_{phr}$) to layout blocks li2025moedit to further enforce alignment between local image regions and their corresponding text semantics, thereby ensuring quantity consistency in text-to-multi-object image generation. For fine-grained control, Adaptive Multi-modal Guidance (AMG) module is employed. Signal Encoder encodes activated signals $\mathbf{C}$ into features ${\mathbf{F}_{i}}$, $i\in\{s,o,b\}$, where ${(\varnothing)}$ represents null embeddings, and further forms a unified feature representation $\mathbf{C}_{unif}$. An Adaptive Controller then generates a structured intent $\mathbf{C}_{str}$, which comprises spatial layout and visual attribute specifications of constraints, and is propagated into U-Net via Intent-Guided Interaction Mechanism. $\mathbf{Q}_{net}$ denotes pre-trained queries in U-Net.
• Figure 4: The construction pipeline of the MoCA benchmark. First, diverse predefined text inputs are used with generative models to produce candidate images. Next, manual screening retains high-quality images with clear object attributes, without requiring quantity consistency between the generated results and the input text. Finally, selected images are manually annotated to establish a multi-dimensional benchmark comprising text descriptions, structure references, object references, and bounding boxes.
• Figure 5: Qualitative comparisons on T → Image. MoGen achieves superior performance in text-to-multi-object image generation, excelling in quantity consistency, image quality, and semantic alignment. FLUX flux2024 ranks second with partial limitations, Omnigen2 wu2025omnigen2 shows moderate accuracy.