DressCode: Autoregressively Sewing and Generating Garments from Text Guidance

TL;DR

DressCode introduces a text-driven 3D garment generation framework that yields sewing patterns and PBR textures from natural language prompts. It combines SewingGPT, a GPT-based autoregressive model that tokenizes sewing patterns, with a diffusion-based texture generator fine-tuned for tile-based textures, enabling CG-ready garments, pattern completion, and texture editing. Key contributions include a novel sewing-pattern quantization scheme, cross-attention conditioned generation from CLIP prompts, and a two-stage texture synthesis pipeline that supports draping on 3D humans and interactive editing. The approach demonstrates superior quality and prompt alignment in experiments and user studies, with practical implications for fashion design, virtual try-on, and digital human creation, while acknowledging dataset and ethical limitations.

Abstract

Apparel's significant role in human appearance underscores the importance of garment digitalization for digital human creation. Recent advances in 3D content creation are pivotal for digital human creation. Nonetheless, garment generation from text guidance is still nascent. We introduce a text-driven 3D garment generation framework, DressCode, which aims to democratize design for novices and offer immense potential in fashion design, virtual try-on, and digital human creation. We first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns with text guidance. We then tailor a pre-trained Stable Diffusion to generate tile-based Physically-based Rendering (PBR) textures for the garments. By leveraging a large language model, our framework generates CG-friendly garments through natural language interaction. It also facilitates pattern completion and texture editing, streamlining the design process through user-friendly interaction. This framework fosters innovation by allowing creators to freely experiment with designs and incorporate unique elements into their work. With comprehensive evaluations and comparisons with other state-of-the-art methods, our method showcases superior quality and alignment with input prompts. User studies further validate our high-quality rendering results, highlighting its practical utility and potential in production settings. Our project page is https://IHe-KaiI.github.io/DressCode/.

Figures (15)

• Figure 1: Overview of our SewingGPT pipeline. We quantize sewing patterns to the sequence of tokens and adopt a GPT-based architecture to generate the tokens autoregressively. Our SewingGPT enables users to generate highly diverse and high-quality sewing patterns under text prompt guidance.
• Figure 2: Details of our quantization. We present an example of a part of a sleeveless dress, including a skirt panel (Panel 1) and a top panel (Panel 2). Assuming $N_1 < N_2 = K$, we require zero-padding for tokens from Panel 1.
• Figure 3: Examples of our data captions. We utilize the rendered images and ask GPT-4V with the designed prompt for detailed captions.
• Figure 4: Overview of our entire DressCode pipeline for customized garment generation. We employ a large language model to obtain shape prompts and texture prompts with natural language interaction and utilize the SewingGPT and a fine-tuned Stable Diffusion for high-quality and CG-friendly garment generation.
• Figure 5: Examples of our multiple garments draping process. Starting with initial sewing patterns, we first drape the inside T-shirt, followed by draping the outside jacket onto the model’s body.