ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians

TL;DR

This work introduces ClotheDreamer, a 3D Gaussian-based method for generating wearable, production-ready 3D garment assets from text prompts, and proposes a novel representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate optimization.

Abstract

High-fidelity 3D garment synthesis from text is desirable yet challenging for digital avatar creation. Recent diffusion-based approaches via Score Distillation Sampling (SDS) have enabled new possibilities but either intricately couple with human body or struggle to reuse. We introduce ClotheDreamer, a 3D Gaussian-based method for generating wearable, production-ready 3D garment assets from text prompts. We propose a novel representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate optimization. DCGS represents clothed avatar as one Gaussian model but freezes body Gaussian splats. To enhance quality and completeness, we incorporate bidirectional SDS to supervise clothed avatar and garment RGBD renderings respectively with pose conditions and propose a new pruning strategy for loose clothing. Our approach can also support custom clothing templates as input. Benefiting from our design, the synthetic 3D garment can be easily applied to virtual try-on and support physically accurate animation. Extensive experiments showcase our method's superior and competitive performance. Our project page is at https://ggxxii.github.io/clothedreamer.

Figures (11)

• Figure 2: Overview of ClotheDreamer. Given a text description, we first leverage ChatGPT to determine clothing ID types for initialization. Our Disentangled Clothe Gaussian Splatting (DCGS) $\mathcal{G}^{'}$ represents clothed avatar as One-Gaussian model but freezes body Gaussian splats to achieve separate supervision. With parsed Gaussian Splatting (GS) render $\mathcal{R}^{'}$, we use Bidreactional SDS to guide clothing and body RGBD renderings separately with pose condition. We also support template mesh input for versatile personalized 3D garment generation.
• Figure 3: Clothing types. We offer six common groups for initializing DCGS in our zero-shot garment generation.
• Figure 4: Importance of One-Gaussian Initialization. Artifacts may appear for clothed avatar rendering with Two-Gaussians model. Please kindly zoom in for a better comparison.
• Figure 5: Improper prune strategy humangaussian may mistakenly eliminate useful Gaussian splats for loose clothing.
• Figure 6: Qualitative comparison of garment generation from text. We compare with recent state-of-the-art 3D generation baselines on seven different garment text descriptions. Note that red text highlights incomplete garment generation, while orange arrows indicate geometry artifacts for redundant human body parts.