GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generation

TL;DR

GarmentX addresses the challenge of generating high-fidelity, wearable 3D garments from a single image by introducing a structured parametric representation and a masked autoregressive diffusion-augmented generator that outputs editable garment parameters. These parameters are decoded into valid sewing patterns via GarmentCode and simulated into 3D garments, ensuring physical validity and wearability. A large GarmentX dataset of 378,682 parameter–image pairs, augmented with canny-conditioned ControlNet renders, enables robust learning and alignment to input images. The approach achieves state-of-the-art geometry fidelity, input-image alignment, and editing capabilities, while offering fast inference and strong generalization, with plans to extend to clothed humans and end-to-end workflows.

Abstract

This work presents GarmentX, a novel framework for generating diverse, high-fidelity, and wearable 3D garments from a single input image. Traditional garment reconstruction methods directly predict 2D pattern edges and their connectivity, an overly unconstrained approach that often leads to severe self-intersections and physically implausible garment structures. In contrast, GarmentX introduces a structured and editable parametric representation compatible with GarmentCode, ensuring that the decoded sewing patterns always form valid, simulation-ready 3D garments while allowing for intuitive modifications of garment shape and style. To achieve this, we employ a masked autoregressive model that sequentially predicts garment parameters, leveraging autoregressive modeling for structured generation while mitigating inconsistencies in direct pattern prediction. Additionally, we introduce GarmentX dataset, a large-scale dataset of 378,682 garment parameter-image pairs, constructed through an automatic data generation pipeline that synthesizes diverse and high-quality garment images conditioned on parametric garment representations. Through integrating our method with GarmentX dataset, we achieve state-of-the-art performance in geometric fidelity and input image alignment, significantly outperforming prior approaches. We will release GarmentX dataset upon publication.

Figures (12)

• Figure 1: GarmentX is an image-guided 3D garment generator that produces editable garment parameters, decodes valid sewing patterns, and simulates wearable 3D garments. By adjusting garment parameters, users can easily modify the shape, style, and even category of the garments. The generated garments are diverse, high-fidelity, and physically plausible.
• Figure 2: Automatic Data Construction Pipeline. We construct garment parameters-image pairs using ControlNet and Blender.
• Figure 3: Overview of GarmentX. Taking a single image as input, GarmentX trains a masked autoregressive generation model directly upon our GarmentX representation, extracts condition image tokens via DINOv2, processes them through MAE encoder-decoder architecture and diffusion MLP. The generated GarmentX representation is projected back to original scale and then reconstruct sewing patterns through GarmentCode, and finally simulate 3D garments wearable on arbitrary human bodies.
• Figure 4: Qualitative comparisons on CLOTH3D dataset. GarmentX produce wearable, open-structure, and complete 3D garments.
• Figure 5: Qualitative comparisons with Trellis. GarmentX generates single-layer, open-structure 3D garments.