Unraveling MMDiT Blocks: Training-free Analysis and Enhancement of Text-conditioned Diffusion

TL;DR

This paper addresses the opaque block-wise interactions within Multimodal Diffusion Transformers (MMDiT) by proposing a systematic removal, disablement, and enhancement framework across SD3.5, FLUX, and Qwen Image. It reveals that semantic information tends to be encoded in earlier blocks while fine details are produced by later blocks; it also finds that disabling textual conditions is more disruptive than removing blocks and that targeted text-enhancement in specific blocks can improve alignment without sacrificing quality. Building on these insights, the authors introduce training-free techniques for strengthening text-visual alignment, enabling precise editing via self-attention injection, and accelerating inference by skipping non-critical blocks, all demonstrated to improve T2I-CompBench++ and GenEval metrics while maintaining synthesis quality. The work provides both fundamental understanding of MMDiT internals and practical, broadly applicable strategies for improving text-conditioned diffusion systems in generation, editing, and speed.

Abstract

Recent breakthroughs of transformer-based diffusion models, particularly with Multimodal Diffusion Transformers (MMDiT) driven models like FLUX and Qwen Image, have facilitated thrilling experiences in text-to-image generation and editing. To understand the internal mechanism of MMDiT-based models, existing methods tried to analyze the effect of specific components like positional encoding and attention layers. Yet, a comprehensive understanding of how different blocks and their interactions with textual conditions contribute to the synthesis process remains elusive. In this paper, we first develop a systematic pipeline to comprehensively investigate each block's functionality by removing, disabling and enhancing textual hidden-states at corresponding blocks. Our analysis reveals that 1) semantic information appears in earlier blocks and finer details are rendered in later blocks, 2) removing specific blocks is usually less disruptive than disabling text conditions, and 3) enhancing textual conditions in selective blocks improves semantic attributes. Building on these observations, we further propose novel training-free strategies for improved text alignment, precise editing, and acceleration. Extensive experiments demonstrated that our method outperforms various baselines and remains flexible across text-to-image generation, image editing, and inference acceleration. Our method improves T2I-Combench++ from 56.92% to 63.00% and GenEval from 66.42% to 71.63% on SD3.5, without sacrificing synthesis quality. These results advance understanding of MMDiT models and provide valuable insights to unlock new possibilities for further improvements.

Figures (7)

• Figure 1: Visual comparison of text-to-image and editing results, demonstrating better text alignment across semantic attributes.
• Figure 2: Systematic analysis overview. We apply removing, disabling, and enhancing to each MMDiT block to analyze their individual and joint effects with textual conditions.
• Figure 3: Block-wise analysis results across various MMDiT-based models on different attributes. We identify each block's specific role in generating images and its interactions with textual conditions. The accuracy of different attributes is evaluated by QwenVL-2.5-72B on multiple runs, and DINOv2 score shows the perceptual similarities.
• Figure 4: Attention map of different tokens. We can enhance specific tokens to boost their impact.
• Figure 5: Qualitative comparisons between baselines and our method. Our method significantly improves the text alignment across various semantic attributes including amount, colors, textures, and complex prompts, etc. Zoom in for details.