DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers

TL;DR

This work defines a novel task—Layer-Wise Decomposition of Alpha-Composited Images—targeting semi-transparent and transparent blending. It introduces AlphaBlend, a large-scale dataset, and DiffDecompose, a diffusion-transformer framework that probabilistically generates Plausible layer decompositions using In-Context Decomposition and Layer Position Encoding Cloning to maintain pixel-level correspondence. The approach outperforms mask-based inpainting baselines on real-world transparent scenarios, with strong ablations demonstrating the importance of LPEC and ICD. The work advances transparent-layer understanding and offers a new benchmark for multi-layer image decomposition under nonlinear blending.

Abstract

Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi-transparent or transparent layer occlusions due to mask prior dependencies, static object assumptions, and the lack of datasets. In this paper, we delve into a novel task: Layer-Wise Decomposition of Alpha-Composited Images, aiming to recover constituent layers from single overlapped images under the condition of semi-transparent/transparent alpha layer non-linear occlusion. To address challenges in layer ambiguity, generalization, and data scarcity, we first introduce AlphaBlend, the first large-scale and high-quality dataset for transparent and semi-transparent layer decomposition, supporting six real-world subtasks (e.g., translucent flare removal, semi-transparent cell decomposition, glassware decomposition). Building on this dataset, we present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image, semantic prompts, and blending type. Rather than regressing alpha mattes directly, DiffDecompose performs In-Context Decomposition, enabling the model to predict one or multiple layers without per-layer supervision, and introduces Layer Position Encoding Cloning to maintain pixel-level correspondence across layers. Extensive experiments on the proposed AlphaBlend dataset and public LOGO dataset verify the effectiveness of DiffDecompose. The code and dataset will be available upon paper acceptance. Our code will be available at: https://github.com/Wangzt1121/DiffDecompose.

Figures (15)

• Figure 1: We propose a novel generative task, Layer-Wise Decomposition of Alpha-Composited Images, to recover constituent layers from single overlapped images under the condition of semi-transparent or transparent layer non-linear occlusion. We introduce the AlphaBlend dataset, the first large-scale dataset for transparent and semi-transparent layer decomposition to support six real-world subtasks. (a) shows generation results on alpha layer removal (I–II), semi-transparent and transparent layer separation (III–IV), and complex non-linear alpha-blend decomposition (V–VI). (b) highlights the dataset's broad coverage across categories e.g., flare, fog, glassware, X-ray contraband.
• Figure 2: The comparison of conventional inpainting methods with our proposed DiffDecompose. The conventional inpainting approach (blue background) relies on predefined object masks and predicts missing regions, often causing semantic errors in transparent scenes. In contrast, DiffDecompose (green background) conditions on the full composited image and jointly predicts foreground and background via layer-level decomposition. This formulation removes the need for explicit masks and enables more accurate separation in the presence of transparency and complex blending.
• Figure 3: The DiffDecompose framework comprises two steps: (1) VAE encodes the source image into a condition token and concatenates it with a noisy latent token, controlling the generation of layer decomposition. (2) In-Context Decomposition constructs an image-conditioned base model to decompose multiple layers. Among them, the LPEC clones the position encoding to ensure the alignment between different layers, and MMAttention are introduced to process the multi-modulation features.
• Figure 4: Our DiffDecompose shows impressive layer-level decomposition results of the image. It can solve the layer-level decomposition (i.e., Subtask II and Subtask III), the nonlinear alpha-blend layer removal and decomposition (i.e., Subtask I and Subtask VI), and the semi-transparent/transparent object-level decomposition (i.e., Subtask IV and Subtask VI), demonstrating its generalization and application in various scenarios.
• Figure 5: Qualitative comparisons between our method and other methods.