Bayesian Differentiable Physics for Cloth Digitalization

TL;DR

This work proposes a new Bayesian differentiable cloth model to estimate the complex material heterogeneity of real cloths and shows it can provide highly accurate digitalization from very limited data samples.

Abstract

We propose a new method for cloth digitalization. Deviating from existing methods which learn from data captured under relatively casual settings, we propose to learn from data captured in strictly tested measuring protocols, and find plausible physical parameters of the cloths. However, such data is currently absent, so we first propose a new dataset with accurate cloth measurements. Further, the data size is considerably smaller than the ones in current deep learning, due to the nature of the data capture process. To learn from small data, we propose a new Bayesian differentiable cloth model to estimate the complex material heterogeneity of real cloths. It can provide highly accurate digitalization from very limited data samples. Through exhaustive evaluation and comparison, we show our method is accurate in cloth digitalization, efficient in learning from limited data samples, and general in capturing material variations. Code and data are available https://github.com/realcrane/Bayesian-Differentiable-Physics-for-Cloth-Digitalization

Figures (8)

• Figure 0: We introduce a Bayesian Differentiable Physics (BPD) model for digitalizing real cloths by inferring their physical properties from the standard Cusick drape data (a-1, b-1, c-1 left). The digitalized cloths exhibit various drapabilities, faithfully reflecting their diverse mechanical characteristics and materials (a-1, b-1, c-1 middle and right). Further, our model enables the generalization of the learned mechanical characteristics and materials to garments (a-2, b-2, c-2).
• Figure 1: Cusick drape testing. The Tester, i.e. Cusick drape meter, has an inner support panel (blue), an outer support panel (red) and a frosted glass lid (green). A round cloth sample is first laid flat on the support panels (light blue in Initial State). Then the outer support panel is lowered to allow the cloth to naturally drape (Drape State). Next, the glass lid is closed so that the light source at the bottom can project the cloth to the lid which is recorded by a camera at the top (Capturing). Finally, the cloth Silhouette is extracted from the Raw Photo. The whole drape meter is in a black chamber so the testing process is not observable. Due to patent restrictions, the images are rendered, not the real device.
• Figure 2: (a) A circular (diameter=$30cm$) cloth sample for Cusick drape test. (b) Cloth sample mesh has 2699 vertices, 7924 edges (7754 bending edges), and 5226 faces. A bending edge (highlighted in orange) is shared between two adjacent triangles, so the edges highlighted in blue (the boundary) are not bending edges.
• Figure 3: (a) The Probabilistic Graphical Model (PGM) of our Bayesian Differentiable Simulator. (b) Model overview. The physical parameters (stretching stiffness, bending stiffness) are first drawn from their learnable posteriors. Then the parameters and the cloth initial state are fed to a differentiable cloth simulator to run and predict cloth's final state $\mathcal{S}_n=\{\mathbf{x}_n, \mathbf{\dot{x}}_n\}$. The cloth in the final state is passed to a differentiable renderer. The rendered cloth silhouette is compared with the ground truth to compute the loss for back-propagation to update the parameters in the posteriors.
• Figure 4: Given the digitialized cloths, our BDP model can simulate the skirts made from these cloths and reflect cloth material heterogeneity and draping stochasticity.