Reconstruction of Manipulated Garment with Guided Deformation Prior

TL;DR

This work leverage the implicit sewing patterns (ISP) model for garment modeling and extend it by adding a diffusion-based deformation prior to represent these shapes, to recover 3D garment shapes from incomplete 3D point clouds acquired when the garment is folded.

Abstract

Modeling the shape of garments has received much attention, but most existing approaches assume the garments to be worn by someone, which constrains the range of shapes they can assume. In this work, we address shape recovery when garments are being manipulated instead of worn, which gives rise to an even larger range of possible shapes. To this end, we leverage the implicit sewing patterns (ISP) model for garment modeling and extend it by adding a diffusion-based deformation prior to represent these shapes. To recover 3D garment shapes from incomplete 3D point clouds acquired when the garment is folded, we map the points to UV space, in which our priors are learned, to produce partial UV maps, and then fit the priors to recover complete UV maps and 2D to 3D mappings. Experimental results demonstrate the superior reconstruction accuracy of our method compared to previous ones, especially when dealing with large non-rigid deformations arising from the manipulations.

Figures (7)

• Figure 1: Recovering the 3D shape of folded and crumpled garments from incomplete point clouds. Top: The input point clouds (green) overlaid on the ground truth meshes (gray). Bottom: Our reconstructions.
• Figure 2: Our framework. Given a point cloud, we first map it to UV space to obtain sparse UV maps $\tilde{\mathcal{M}}$ and panel masks $\tilde{\mathcal{O}}$. We recover complete UV maps $\mathcal{M}$ and panel masks $\mathcal{O}$ from them using ISP and a deformation prior, enabling the reconstruction of the deformed garment's 3D mesh.
• Figure 3: The projected sparse masks $\tilde{\mathcal{O}}$ and UV maps $\tilde{\mathcal{M}}$ of the point clouds with (a) the maximum volume and (b) the minimum volume. The point clouds are color coded by their 3D positions.
• Figure 4: The guided reverse diffusion process. The UV maps of the deformed garment are generated by using the observations from the input point cloud as guidance to direct the reverse diffusion process.
• Figure 5: Qualitative comparisons of our method to GarmentTracking (initialized with ground truth meshes) on VR-Folding dataset for the categories of Pants, Top, Shirt and Skirt.