PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment

TL;DR

PSDiffusion tackles the challenge of synthesizing coherent multi-layer RGBA images by leveraging interaction priors from pretrained RGB diffusion models. It introduces an RGBA VAE, a layer cross-attention reweighting mechanism to guide foreground layout from a global prompt, and a partial joint self-attention module to foster inter-layer coherence. To address data scarcity, it presents Inter-Layer, a 30K high-quality, human-curated dataset with 3–6 layered assets and artist-level alpha mattes. Experiments show PSDiffusion outperforms state-of-the-artLayerDiffuse and ART in layout plausibility, cross-layer interactions, and alpha quality, validated by quantitative metrics and a user study.

Abstract

Transparent image layer generation plays a significant role in digital art and design workflows. Existing methods typically decompose transparent layers from a single RGB image using a set of tools or generate multiple transparent layers sequentially. Despite some promising results, these methods often limit their ability to model global layout, physically plausible interactions, and visual effects such as shadows and reflections with high alpha quality due to limited shared global context among layers. To address this issue, we propose PSDiffusion, a unified diffusion framework that leverages image composition priors from pre-trained image diffusion model for simultaneous multi-layer text-to-image generation. Specifically, our method introduces a global layer interaction mechanism to generate layered images collaboratively, ensuring both individual layer quality and coherent spatial and visual relationships across layers. We include extensive experiments on benchmark datasets to demonstrate that PSDiffusion is able to outperform existing methods in generating multi-layer images with plausible structure and enhanced visual fidelity.

Figures (8)

• Figure 1: Given a complex prompt, our model, PSDiffusion, produces plausible per-layer layouts for each instance and achieves harmonious inter-layer interactions. Compared to the state-of-the-art zhang2024transparent, PSDiffusion generates layered images with more plausible spatial arrangement and more consistent appearances with realistic visual effects (e.g., reflections for the boat and swan), without manual spatial refinement. [Best viewed with zoom-in]
• Figure 2: Visual results of PSDiffusion. Through the design of Layer Cross-Attention Reweighting Module and Partial Joint Self-Attention Mechanism, PSDiffusion synthesize multi-layer transparent images with plausible layout arrangements and harmonious appearance.
• Figure 3: Architecture of PSDiffusion. Our framework processes multi-layer compositions through three key components: (1) A transparent VAE encoder preserving alpha channels; (2) Layer cross-attention reweighting for layout plausibility; (3) Partial joint self-attention for inter-layer context modeling.
• Figure 4: Overview of our global-layer interactive mechanism, composed of layer cross-attention reweighting module and partial joint self-attention module. Layer cross-attention reweighting module extracts the cross-attention map from the global branch to reweight the cross-attention map of the layer branch, guiding the position of foreground layers. Partial joint self-attention module implements a shared attention across global branch and layer branch to facilitate context-aware feature modeling.
• Figure 5: Examples of our proposed Inter-Layer dataset.