EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

Abstract

In this work, we present EchoGen, a unified framework for layout-to-image generation and image grounding, capable of generating images with accurate layouts and high fidelity to text descriptions (e.g., spatial relationships), while grounding the image robustly at the same time. We believe that image grounding possesses strong text and layout understanding abilities, which can compensate for the corresponding limitations in layout-to-image generation. At the same time, images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding. Jointly training both tasks within a unified model can promote performance improvements for each. However, we identify that this joint training paradigm encounters several optimization challenges and results in restricted performance. To address these issues, we propose progressive training strategies. First, the Parallel Multi-Task Pre-training (PMTP) stage equips the model with basic abilities for both tasks, leveraging shared tokens to accelerate training. Next, the Dual Joint Optimization (DJO) stage exploits task duality to sequentially integrate the two tasks, enabling unified optimization. Finally, the Cycle RL stage eliminates reliance on visual supervision by using consistency constraints as rewards, significantly enhancing the model's unified capabilities via the GRPO strategy. Extensive experiments demonstrate state-of-the-art results on both layout-to-image generation and image grounding benchmarks, and reveal clear synergistic gains from optimizing the two tasks together.

Figures (5)

• Figure 1: Our unified approach faithfully parses complex prompts—e.g., color attributes and relative spatial relations—and, combined with layout conditions, achieves more accurate semantic alignment. In contrast, specialized methods (e.g., Gligen and MIGC) often fail to handle such constraints. For instance, in the first row of MIGC results, the model generates a vertical spatial relation (top–middle–bottom), whereas the prompt specifies a depth-based relation (front–middle–back).
• Figure 2: Overview of EchoGen. Left—Parallel Multi‑task Pre‑training: a unified autoregressive transformer is trained on layout‑to‑image and image grounding tasks in parallel, yielding fast acquisition of base capabilities. Middle—Dual Joint Optimization: the image tokens generated by the generation forward pass are directly reused as the input to grounding, forming a single joint objective, which strengthens layout$\to$image$\to$layout cycle consistency. Right—Cycled RL: we execute the layout$\to$image$\to$layout loop and treat the discrepancy between input and recovered layouts as a continuous reward, enabling self-supervised Reinforcement Learning without explicit supervision of intermediate visual outputs.
• Figure 3: Attention Mask of parallel multi-task pre-training.
• Figure 4: Quantitative comparison on layout-to-image generation. The first column lists the global text prompts; the second column visualizes the layout conditions, rendered as (referring-expression, bounding-box) pairs. We highlight a subset of negatives and positives with red and green boxes; note that negatives may correspond to incorrect spatial locations or attributes (e.g., color).
• Figure 5: Ablation on stage transition points. Left: gains when switching from Stage 1$\!\rightarrow$Stage 2; Right: gains when switching from Stage 2$\!\rightarrow$Stage 3. The x-axis records the $\mathrm{AP}$ at entry to the current stage (determined by the preceding stage’s training budget), while the y-axis reports the $\mathrm{AP}$ improvement achieved by that stage.