Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

Shengqiong Wu, Bobo Li, Xinkai Wang, Xiangtai Li, Lei Cui, Furu Wei, Shuicheng Yan, Hao Fei, Tat-seng Chua

TL;DR

The interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations, is introduced, highlighting AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation.

Abstract

Unified Vision-Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we introduce the interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design a two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by reinforcement learning to promote adaptive and autonomous control. Extensive experiments demonstrate that AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, with strong transferability to various UVLMs architectures. Visual analyses further validate the effectiveness of implicit visual thoughts. These results highlight AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation. The project page is at https://sqwu.top/AD-Loop.

Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

TL;DR

The interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations, is introduced, highlighting AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation.

Abstract

Unified Vision-Language Models (UVLMs) aim to advance multimodal learning by supporting both understanding and generation within a single framework. However, existing approaches largely focus on architectural unification while overlooking the need for explicit interaction between the two capabilities during task solving. As a result, current models treat understanding and generation as parallel skills rather than synergistic processes. To achieve real synergy, we introduce the interleaved Analyzing-Drafting problem-solving loop (AD-Loop), a new think paradigm that dynamically alternates between analytic and drafting operations. By interleaving textual thoughts with visual thoughts, AD-Loop enables models to iteratively refine both comprehension and outputs, fostering genuine synergy. To train this mechanism, we design a two-stage strategy: supervised learning on interleaved thought data to initialize alternation, followed by reinforcement learning to promote adaptive and autonomous control. Extensive experiments demonstrate that AD-Loop consistently improves performance across standard benchmarks for both understanding and generation, with strong transferability to various UVLMs architectures. Visual analyses further validate the effectiveness of implicit visual thoughts. These results highlight AD-Loop as a principled and broadly applicable strategy for synergizing comprehension and creation. The project page is at https://sqwu.top/AD-Loop.
Paper Structure (52 sections, 10 equations, 17 figures, 6 tables)

This paper contains 52 sections, 10 equations, 17 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Model Architecture
  5. Stage-1: Supervised Training of Interleaved Thinking
  6. Dataset Construction.
  7. Training.
  8. Stage-2: RL-driven Adaptive Interleaved Thinking
  9. Adaptive Interleaved Thinking.
  10. Reward Assignment.
  11. Optimization.
  12. Experiments
  13. Settings
  14. Dataset.
  15. Implementations.
  16. ...and 37 more sections

Figures (17)

  • Figure 1: Illustration of the interleaved analytic–drafting problem-solving loop, where understanding and generation interact synergistically to yield accurate solutions.
  • Figure 2: Pipeline of our training framework. Stage-1: train the UVLM to emit interleaved thinking traces. Stage-2: apply GRPO for hybrid reasoning. The policy samples multiple traces with/without the interleaved AD-Loop for each input. A reward model scores outcomes, and then group-normalized advantages are applied to update the policy, teaching the model when AD-Loop helps.
  • Figure 3: Qualitative comparison: original prompt (left), self-think mode, interleaved thoughts, and text-only thoughts filtered from the interleaved thoughts (right).$[$V-T$]$ means latent visual thoughts.
  • Figure 4: Absolute performance gain after applying AD-Loop, comparing Janus-Pro with discrete tokenization and BAGEL with continuous embedding for visual thoughts learning.
  • Figure 5: Examples of latent visual thoughts. Each case shows the original image (left) and the corresponding visual thoughts (right), capturing abstract visual structures.
  • ...and 12 more figures