Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis

TL;DR

This work tackles the challenge of turning multi-modal design ideas into manufacturable, geometry-accurate sewing patterns. It introduces Design2GarmentCode, a neurosymbolic pipeline that combines a pre-trained Large Multimodal Model as a design interpreter (MMUA) with a finetuned LLM (DSL-GA) that synthesizes GarmentCode parametric garment programs, executed by the GarmentCode engine. A lightweight projector Ψ converts MMUA-derived design features into a fixed-length vector of 122 design parameters, enabling centimeter-level precision and compact representation. Experiments across text-, image-, and sketch-guided inputs show superior simulation feasibility, prompt alignment, and pattern diversity compared with DressCode and SewFormer, while reducing data and computation needs. The approach offers a practical, scalable path toward flexible, production-ready garment design that integrates with existing CAD and simulation tools.

Abstract

Sewing patterns, the essential blueprints for fabric cutting and tailoring, act as a crucial bridge between design concepts and producible garments. However, existing uni-modal sewing pattern generation models struggle to effectively encode complex design concepts with a multi-modal nature and correlate them with vectorized sewing patterns that possess precise geometric structures and intricate sewing relations. In this work, we propose a novel sewing pattern generation approach \textbf{Design2GarmentCode} based on Large Multimodal Models (LMMs), to generate parametric pattern-making programs from multi-modal design concepts. LMM offers an intuitive interface for interpreting diverse design inputs, while pattern-making programs could serve as well-structured and semantically meaningful representations of sewing patterns, and act as a robust bridge connecting the cross-domain pattern-making knowledge embedded in LMMs with vectorized sewing patterns. Experimental results demonstrate that our method can flexibly handle various complex design expressions such as images, textual descriptions, designer sketches, or their combinations, and convert them into size-precise sewing patterns with correct stitches. Compared to previous methods, our approach significantly enhances training efficiency, generation quality, and authoring flexibility.

Figures (19)

• Figure 1: Traditional sewing pattern generation approaches (top) use uni-modal models trained on synthetic datasets generated by parametric pattern-making programs (red arrow) to convert text or image prompts into vector-quantized patterns. These methods are resource-intensive and often yield oversimplified patterns with stitching errors. Our approach (bottom) utilizes large pre-trained LMMs to directly translate design concepts into parametric programs and configuration files (blue arrow), enabling dedicated, structurally correct pattern generation from multi-modal design inputs within a unified framework.
• Figure 2: (a) Despite prompts specifying diverse neckline types, DressCode he2024dresscode consistently produces only V-neck designs, indicating limited generation diversity. (b) SewFormer liu2023sewformer often generates sewing patterns with self-intersecting panels, compromising pattern validity. (c) Stitching errors are also prevalent in Sewformer liu2023sewformer, as shown here where a pant side seam is mistakenly stitched to a shirt shoulder seam, resulting in draping failure.
• Figure 3: Overview of Dress2GarmentCode. (1) Program Learning: we finetune the DSL Generation Agent (DSL-GA) using GarmentCode example programs, teaching it the GarmentCode grammar and the semantics of each design parameter. (2) Prompt Synthesis: the DSL-GA generates prompts for the Multi-Modal Understanding Agent (MMUA) to interpret and extract relevant design features from the input (3). (4) Program Synthesis: based on the MMUA's responses, the DSL-GA synthesizes GarmentCode-compliant design configurations and garment programs, which are then executed by the GarmentCode engine to produce sewing patterns and simulated garments (5). To enhance robustness, we incorporate two validation loops: during program synthesis, we employ rule-based validations (7) to ensure the MMUA's outputs are sufficient for generating complete and valid garment programs and design parameters; after the initial generation, the MMUA compares the generated design with the input and suggests modifications to minimize discrepancies.
• Figure 4: Quality Comparison on Text-Guided Sewing Pattern Generation. For each design, we present the generated pattern using our method (left) alongside DressCode he2024dresscode (right), including front and back renderings of the draped garment. We highlight design elements accurately captured by our method but missed by DressCode he2024dresscode use red color in the input prompt.
• Figure 5: Quality Comparison on Image-Guided Sewing Pattern Generation. We compare our method with Sewformer liu2023sewformer on Internet-collected fashion photographs (left), and AI-generated design images without human models (right). The results indicate that our method successfully captures design details from diverse styles, producing sewing patterns that accurately reflect neckline (a, d), cuffs (a, e, g), darts (c, d), and asymmetry (f). In contrast, Sewformer’s results exhibit several issues, including incorrect necklines (a, d), missing components (b, g), misplaced or imaginary stitches (d, e), and extraneous pattern pieces (h). Additionally, since Sewformer’s pattern generation does not account for body shape, garments like skirts and pants frequently appear oversized around the waist, causing them to sag when draped.