GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details

TL;DR

GarmentDreamer introduces a $3DGS$-guided framework to synthesize high-fidelity, simulation-ready 3D garments from text prompts. A warm-start garment template is refined through a coarse-to-fine geometry deformer guided by normal and RGBA cues, and texture is reconstructed with an implicit Neural Texture Field optimized via Score Distillation Sampling. The method achieves improved multi-view consistency and texture detail while producing wearable openings, outperforming state-of-the-art deformation-based and image-driven baselines in both quantitative and qualitative evaluations. This approach enables efficient generation of diverse, realistic garments suitable for downstream simulation, animation, and virtual try-on tasks.

Abstract

Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and texturing, which are time-consuming and costly. Recent advances in diffusion-based generative models have enabled new possibilities for 3D garment generation from text prompts, images, and videos. However, existing methods either suffer from inconsistencies among multi-view images or require additional processes to separate cloth from the underlying human model. In this paper, we propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate wearable, simulation-ready 3D garment meshes from text prompts. In contrast to using multi-view images directly predicted by generative models as guidance, our 3DGS guidance ensures consistent optimization in both garment deformation and texture synthesis. Our method introduces a novel garment augmentation module, guided by normal and RGBA information, and employs implicit Neural Texture Fields (NeTF) combined with Score Distillation Sampling (SDS) to generate diverse geometric and texture details. We validate the effectiveness of our approach through comprehensive qualitative and quantitative experiments, showcasing the superior performance of GarmentDreamer over state-of-the-art alternatives. Our project page is available at: https://xuan-li.github.io/GarmentDreamerDemo/.

Figures (7)

• Figure 1: Starting with text prompts, we generate a garment template mesh using a diffusion model. We then optimize a 3DGS from the template and text. Using RGBA, normal maps, and masks, we guide a two-stage deformation to refine the mesh into the final shape. Finally, we reconstruct and optimize an implicit texture field via vsd, producing high-quality textured garment meshes that can be applied to downstream simulation/animation tasks.
• Figure 2: Garment Gallery. We showcase a gallery of textured garment meshes of different clothing categories generated by GarmentDreamer. We refer to the supplementary material for closer observations of these garments and more generation results.
• Figure 3: We use the SMPL-X mesh sequence to drive the dynamics of our generated garments. The utilization of CIPC resolves frictional collisions and self-collisions effectively and guarantees non-penetrative results in garment simulations.
• Figure 4: Normal Comparisons. We visualize the normal maps for a better comparison of garment geometry between GarmentDreamer and other methods. Our proposed method generates visually plausible garment meshes, featuring finer geometric details such as natural wrinkles and smooth boundaries.
• Figure 5: Qualitative Comparisons. While baseline methods either produce unrealistic geometric artifacts, e.g. spikes and excessive smoothness, or non-garment textures, GarmentDreamer excels in generating high-quality, simulation-ready non-watertight garments with detailed textures and fine wrinkle details.