Neural Garment Dynamic Super-Resolution

TL;DR

Neural Garment Dynamic Super-Resolution (GDSR) presents a lightweight mesh-based pipeline to elevate low-resolution garment simulations to high-resolution detail. It encodes coarse dynamics and garment–body interactions into super-resolution features via a Mesh-Graph-Net, then uses a hyper-network to generate per-triangle wrinkle residuals while a separate decoder corrects the coarse geometry, operating in a roll-out framework for temporal consistency. The approach generalizes well to unseen motions, body shapes, and garment types, achieves notable improvements over baselines like DDE and PhysGraph, and runs efficiently on modest hardware with a compact model (~65 MB). Limitations include friction-driven wrinkle capture and some inter-layer collision handling, with future work aimed at richer physics signals, improved collision resolution, and rendering integration.

Abstract

Achieving efficient, high-fidelity, high-resolution garment simulation is challenging due to its computational demands. Conversely, low-resolution garment simulation is more accessible and ideal for low-budget devices like smartphones. In this paper, we introduce a lightweight, learning-based method for garment dynamic super-resolution, designed to efficiently enhance high-resolution, high-frequency details in low-resolution garment simulations. Starting with low-resolution garment simulation and underlying body motion, we utilize a mesh-graph-net to compute super-resolution features based on coarse garment dynamics and garment-body interactions. These features are then used by a hyper-net to construct an implicit function of detailed wrinkle residuals for each coarse mesh triangle. Considering the influence of coarse garment shapes on detailed wrinkle performance, we correct the coarse garment shape and predict detailed wrinkle residuals using these implicit functions. Finally, we generate detailed high-resolution garment geometry by applying the detailed wrinkle residuals to the corrected coarse garment. Our method enables roll-out prediction by iteratively using its predictions as input for subsequent frames, producing fine-grained wrinkle details to enhance the low-resolution simulation. Despite training on a small dataset, our network robustly generalizes to different body shapes, motions, and garment types not present in the training data. We demonstrate significant improvements over state-of-the-art alternatives, particularly in enhancing the quality of high-frequency, fine-grained wrinkle details.

Figures (14)

• Figure 1: GDSR overview. With the low-resolution garment dynamics $C_{t-2}$, $C_{t-1}$ and $C_t$, the underlying body motion $B_{t-1}$, $B_t$, and the previous prediction $\hat{C}_{t-2}$, $\hat{C}_{t-1}$, our GDSR predicts the displacements $D_t$ to compute the corrected coarse garment shape $\hat{C}_t$, and jointly generates detail wrinkle residual $R$ to produce the final high-resolution garment geometry $G_t$.
• Figure 2: Garment dynamic super-resolution architecture. Our method operates in a roll-out prediction manner. We represent the inputs of low-resolution simulations, body motions, and downsampled previous predictions as a graph with node features $Q_t$ and edge features $E_t$. First, we employ Mesh-Graph-Net$\mathcal{H}$ to compute super-resolution features $z_t:=[\hat{z}_t | \tilde{z}_t]$ for coarse garment mesh. Considering the impact of coarse garment shape on detail wrinkle performance, Decoder$\mathcal{E}$ decodes features $\hat{z}_t$ to correct the low-resolution garment shape, resulting in $\hat{C}_t$. Simultaneously, Hyper-Net$\mathcal{A}$ exploits super-resolution features $\tilde{z}_t$ to construct an implicit function $W_t^f$ of detail wrinkle residuals for each mesh triangle. Finally, we generate the high-resolution garment $G_t$ by applying the detail wrinkle residuals on the corrected coarse garment.
• Figure 3: Explicit collision handling. We detect the garment collisions with the body and that between the fabric layers at a coarse level of garment geometry, and propagate the detection to efficiently resolve the collisions for high resolution garment.
• Figure 4: We run our method by roll-out prediction on both seen (1st row) and unseen (2nd row) motions with the training garment types. For each example, we show input of low resolution simulation, results of coarse shape correction and detailed high resolution results, as well as the reference of high resolution simulation.
• Figure 5: We evaluate the generalization of our method on motions and garment types that are both out of the training data. Our method can synthesize fine-grained wrinkle details for either single- or multi-layer garments.