Image2Garment: Simulation-ready Garment Generation from a Single Image

TL;DR

Image2Garment addresses the challenge of generating simulation-ready garments from a single image by introducing a semantically grounded two-stage latent decomposition: first infer material descriptors $M$ and auxiliary attributes $Z$, then map these to physically valid simulator parameters via a learned function $f_\phi$. A two-dataset framework, FTAG for fabric attributes and T2P for material-to-physics mappings, enables data-efficient training of a fully feed-forward pipeline that yields both garment geometry $S$ and physics $\boldsymbol{\theta}$ without iterative optimization. The approach combines a fine-tuned vision-language model (Qwen-2.5VL with LoRA) for attribute prediction, a density–thickness estimator based on hierarchical retrieval, and Random Forest regressors to predict physics parameters, with geometry provided by ChatGarment. Experiments on synthetic 4D-Dress data and in-the-wild sequences show state-of-the-art accuracy in fabric attribute estimation and physics prediction, delivering faster, scalable, and more faithful dynamic draping than optimization-based baselines, thereby enabling practical image-to-simulation workflows for VR, gaming, and fashion design. The work also provides comprehensive benchmarks and datasets that facilitate future research in data-efficient, simulation-ready garment generation from single images.

Abstract

Estimating physically accurate, simulation-ready garments from a single image is challenging due to the absence of image-to-physics datasets and the ill-posed nature of this problem. Prior methods either require multi-view capture and expensive differentiable simulation or predict only garment geometry without the material properties required for realistic simulation. We propose a feed-forward framework that sidesteps these limitations by first fine-tuning a vision-language model to infer material composition and fabric attributes from real images, and then training a lightweight predictor that maps these attributes to the corresponding physical fabric parameters using a small dataset of material-physics measurements. Our approach introduces two new datasets (FTAG and T2P) and delivers simulation-ready garments from a single image without iterative optimization. Experiments show that our estimator achieves superior accuracy in material composition estimation and fabric attribute prediction, and by passing them through our physics parameter estimator, we further achieve higher-fidelity simulations compared to state-of-the-art image-to-garment methods.

Figures (13)

• Figure 1: Image2Garment is a feedforward framework that generates simulation-ready garments from a single image. For this purpose, garment geometry and physical fabric parameters are jointly predicted and used to simulate garment dynamics.
• Figure 2: Impact of garment fabric parameters on simulation. We visualize the final frame of a jumping animation for four different fabrics (wool, cork--cotton, polyester, and a random material) each starting from the exact same initial condition. The choice of garment physics parameters changes the dynamics of the animation drastically. In turn, this makes it critical to estimate these parameters accurately in image-to-garment generation settings to faithfully predict shape, appearance, and dynamics of a garment from visual observations.
• Figure 3: Overview of the Image2Garment pipeline. From a single image, we first generate the garment sewing pattern using ChatGarment bian2025chatgarmentgarmentestimationgeneration. Then we predict fabric attributes such as material composition, fabric family, structure type, weight and thickness aligned with standardized commercial garment tags. Finally, fabric physics parameters are estimated from the predicted attributes, following dominguezelvira2024MechFromMet, yielding mechanically interpretable quantities that describe fabric deformation. The garment geometry and physical parameters are then used to produce simulation-ready garments and physically accurate draping animations for any given motion sequence or body poses such as SMPL SMPL:2015.
• Figure 4: Qualitative comparison on the jumping jack example. The left most column shows the input frame presented to each method and the columns on the right show the animated and rendered garments at different points in time of the animation. Our method's result most closely resemble the ground truth (top row).
• Figure 5: Qualitative comparison on in-the-wild video. The top row shows the original sequence. We only use the left-most frame as input. The rows below show renderings of the garments after simulation.