Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Garment Recovery with Shape and Deformation Priors

Ren Li, Corentin Dumery, Benoît Guillard, Pascal Fua

TL;DR

A fitting approach that utilizes shape and deformation priors learned from synthetic data to accurately capture garment shapes and deformations, including large ones is introduced.

Abstract

While modeling people wearing tight-fitting clothing has made great strides in recent years, loose-fitting clothing remains a challenge. We propose a method that delivers realistic garment models from real-world images, regardless of garment shape or deformation. To this end, we introduce a fitting approach that utilizes shape and deformation priors learned from synthetic data to accurately capture garment shapes and deformations, including large ones. Not only does our approach recover the garment geometry accurately, it also yields models that can be directly used by downstream applications such as animation and simulation.

Garment Recovery with Shape and Deformation Priors

TL;DR

A fitting approach that utilizes shape and deformation priors learned from synthetic data to accurately capture garment shapes and deformations, including large ones is introduced.

Abstract

While modeling people wearing tight-fitting clothing has made great strides in recent years, loose-fitting clothing remains a challenge. We propose a method that delivers realistic garment models from real-world images, regardless of garment shape or deformation. To this end, we introduce a fitting approach that utilizes shape and deformation priors learned from synthetic data to accurately capture garment shapes and deformations, including large ones. Not only does our approach recover the garment geometry accurately, it also yields models that can be directly used by downstream applications such as animation and simulation.
Paper Structure (22 sections, 9 equations, 6 figures, 2 tables)

This paper contains 22 sections, 9 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Tight-Fitting Clothing.
  4. Loose-Fitting Clothing.
  5. Method
  6. ISP Garment Model
  7. Formalization.
  8. Training.
  9. Skinning.
  10. Modeling Large Deformations
  11. From 2D maps to 3D mesh.
  12. Fitting the Models to Images
  13. Experiments
  14. Implementation Details
  15. Dataset, Evaluation Metrics, and Baseline
  16. ...and 7 more sections

Figures (6)

  • Figure 2: Framework. Given an image, (1) we first estimate the normal map of the target garment and the SMPL body parameters $(\beta,\theta)$, which are used to compute the body part segmentation and position maps. (2) The maximum coverage garment shape $\bar{\mathcal{M}}$ is then skinned to closely fit to the body, yielding $\mathcal{M}$. Leveraging (3) pixel-aligned image features, our deformation model (4) predicts occupancy and position maps to correct $\mathcal{M}$ for large deformations. (5) The 3D garment mesh is recovered using ISP and further refined.
  • Figure 3: Cutting and flattening. (a) The front (top) and back (bottom) surfaces after cutting. (b) The flattened panels and UV maps generated by ISP for (a). (c) The maximum-coverage UV maps and its represented 3D shape.
  • Figure 4: Fitting results. Given (a) the normal estimation of an in-the-wild image, (b) is the inference result with ISP recovered geometry. (c) is obtained by optimizing the parameters of the pre-trained deformation model. Further refinement of the mesh vertex positions yields (d).
  • Figure 5: Comparison against SOTA methods. From left to right, we show the input image and the 3D garment meshes recovered by our method and SOTA methods: BCNet, SMPLicit, ClothWild, DrapeNet, ISP. (Since skirt is unavailable for DrapeNet, a random pair of trousers is put on its result in the second row.)
  • Figure 6: Fitting strategy comparison between (a) our fitting method and (b) only vertex optimization.
  • ...and 1 more figures