FashionComposer: Compositional Fashion Image Generation

TL;DR

FashionComposer tackles flexible compositional fashion image generation by supporting multi-modal inputs and an asset library for in-pass multi-asset generation. It introduces subject-binding attention to semantically bind appearance features from multiple references to text prompts, and employs a reference UNet to preserve fine garment details during generation. The method also features correspondence-aware attention and latent code alignment to produce consistent human albums across views. Empirical results show strong performance for multi-garment composition and competitive virtual try-on capabilities, with ablations confirming the value of the reference UNet and binding strategies.

Abstract

We present FashionComposer for compositional fashion image generation. Unlike previous methods, FashionComposer is highly flexible. It takes multi-modal input (i.e., text prompt, parametric human model, garment image, and face image) and supports personalizing the appearance, pose, and figure of the human and assigning multiple garments in one pass. To achieve this, we first develop a universal framework capable of handling diverse input modalities. We construct scaled training data to enhance the model's robust compositional capabilities. To accommodate multiple reference images (garments and faces) seamlessly, we organize these references in a single image as an "asset library" and employ a reference UNet to extract appearance features. To inject the appearance features into the correct pixels in the generated result, we propose subject-binding attention. It binds the appearance features from different "assets" with the corresponding text features. In this way, the model could understand each asset according to their semantics, supporting arbitrary numbers and types of reference images. As a comprehensive solution, FashionComposer also supports many other applications like human album generation, diverse virtual try-on tasks, etc.

Figures (7)

• Figure 1: Overall pipeline of FashionComposer. FashionComposer takes garments composition and optional face, text prompt, and a densepose map projected from SMPL as inputs. The text prompt is encoded and fused with UNets through cross-attention and subject-binding attention, while the garment features are extracted and injected for denoising through Feature Injection Attention.
• Figure 2: Qualitative comparison with multi-reference customization methods, including Emu2 sun2023generative, Collage Diffusion sarukkai2024collage, Paint by Example yang2023paint and AnyDoor chen2024AnyDoor.
• Figure 3: Qualitative comparison with garment-centric fashion image synthesis methods, including StableGarment wang2024StableGarment, IMAGDressing-v1 shen2024IMAGDressing-v1, and Magic Clothing chen2024MagicClothing, where ours better preserves the identity of the target objects. Note that all approaches do not finetune the model on the test samples.
• Figure 4: Diverse virtual try-on results of FashionComposer for upper, lower, and outfit try-on tasks.
• Figure 5: Qualitative comparison for the reference encoder. Reference UNet better preserves the fine details of the garments.