SplitFlux: Learning to Decouple Content and Style from a Single Image

TL;DR

<3-5 sentence high-level summary>SplitFlux analyzes the Flux diffusion model to identify which blocks control content versus style, then introduces two components—Rank-Constrained Adaptation and Visual-Gated LoRA—to disentangle content from style and re-embed disentangled content into new contexts. RCA constrains low-rank updates at semantic boundary blocks to preserve identity and structure, while VGRA uses saliency-guided routing to assign high-rank content updates to foreground regions and low-rank residuals to details, with a complementary loss to promote diverse representations. Experimental results demonstrate improved content preservation and competitive stylization across settings, outperforming prior LoRA-based methods in both disentanglement and recontextualization tasks with greater efficiency. Overall, SplitFlux provides a practical, parameter-efficient framework for single-image content–style disentanglement in diffusion-based generation.

Abstract

Disentangling image content and style is essential for customized image generation. Existing SDXL-based methods struggle to achieve high-quality results, while the recently proposed Flux model fails to achieve effective content-style separation due to its underexplored characteristics. To address these challenges, we conduct a systematic analysis of Flux and make two key observations: (1) Single Stream Blocks are essential for image generation; and (2) Early single stream blocks mainly control content, whereas later blocks govern style. Based on these insights, we propose SplitFlux, which disentangles content and style by fine-tuning the single stream blocks via LoRA, enabling the disentangled content to be re-embedded into new contexts. It includes two key components: (1) Rank-Constrained Adaptation. To preserve content identity and structure, we compress the rank and amplify the magnitude of updates within specific blocks, preventing content leakage into style blocks. (2) Visual-Gated LoRA. We split the content LoRA into two branches with different ranks, guided by image saliency. The high-rank branch preserves primary subject information, while the low-rank branch encodes residual details, mitigating content overfitting and enabling seamless re-embedding. Extensive experiments demonstrate that SplitFlux consistently outperforms state-of-the-art methods, achieving superior content preservation and stylization quality across diverse scenarios.

Figures (8)

• Figure 1: (A) Flux framework. (B) Prompt replacement. $P_1$ and $P_2$ denote distinct prompts, with $P_1^{*}$ being a replication of $P_1$. Replacing $P_1^{*}$ with $P_2$ in a specific block is used to examine the functional contribution of that block. (C) Results of prompt replacement in different blocks. B1–19 denotes injecting $P_2$ into all blocks from Block 1 through Block 19, inclusive.
• Figure 2: Overview of our method. (A) We train the content and style blocks using two different prompts, enabling the model to disentangle image content and style. We design Rank-Constrained Adaptation (RCA) to preserve content identity and Visual-Gated LoRA (VGRA) to allow the disentangled content to be re-embedded into new contexts. (B) The content and style of an image can be obtained simply by loading the corresponding LoRA. (C) For Recontextualization, we load only $\Delta W_{cnt}$ in the content block.
• Figure 3: Results of applying block-specific LoRAs for content disentanglement.
• Figure 4: Qualitative comparison for disentanglement. Our results show that our method achieves superior content and style disentanglement, with the disentangled content preserving the original identity and structure better than other methods.
• Figure 5: Comparison of different methods for Recontextualization. Our approach flexibly adapts to varying contexts while accurately preserving both the subject and style.