PaNDaS: Learnable Deformation Modeling with Localized Control

TL;DR

PaNDaS addresses partial non-rigid deformations by learning a local per-face feature field on a neutral pose and a global pose encoding of the target. A deformation generator predicts a per-face Jacobian field, which is integrated via a Poisson solve to produce a vertex displacement, enabling localized control and partial interpolations without inference-time optimization. The method supports pose mixing, partial pose transfer, and statistical analysis of deformations, demonstrated across MANO, DFAUST, and COMA with state-of-the-art locality and non-linear deformation handling. This has practical impact for realistic avatar deformation, animation, and shape statistics on unregistered data, with robust performance under remeshing and partial descriptions.

Abstract

Non-rigid shape deformations pose significant challenges, and most existing methods struggle to handle partial deformations effectively. We propose to learn deformations at the point level, which allows for localized control of 3D surface meshes, enabling Partial Non-rigid Deformations and interpolations of Surfaces (PaNDaS). Unlike previous approaches, our method can restrict the deformations to specific parts of the shape in a versatile way. Moreover, one can mix and combine various poses from the database, all while not requiring any optimization at inference time. We demonstrate state-of-the-art accuracy and greater locality for shape reconstruction and interpolation compared to approaches relying on global shape representation across various types of human surface data. We also demonstrate several localized shape manipulation tasks and show that our method can generate new shapes by combining different input deformations. Code and data will be made available after the reviewing process.

Figures (15)

• Figure 1: We present PaNDAS: a robust learning-based approach to learn non-rigid deformations of triangular meshes applied to human body surfaces. By combining per-face learned features on a neutral pose mesh (in blue) with a global encoding of a deformed mesh (in purple), the model enables localized control of deformations (in cyan) with user-chosen regions (masked in red on the left). Notably, this approach permits several applications such as mixing poses, the transfer of partial pose, interpolations and localized shape statistics.
• Figure 2: Shape interpolation between unregistered hand meshes. The top row shows a predicted interpolation sequence from a hand with missing ring finger. The bottom row shows an interpolation between two raw scans.
• Figure 3: Overview of the proposed architecture. A pair of meshes is given as input: a source pose $\mathcal{X}$ and a target pose $\mathcal{Y}$. Per-triangle features $(f_t)_t$ are learned on the source mesh and are enriched with a global encoding $z$ of the target mesh to obtain a feature field $(\Tilde{f}_t)_t$ over the source mesh. Finally, the deformation generator learns a Jacobian field from the feature field over the neutral mesh to predict a first-order, per-triangle displacement field $(v_i)_{1\leq i \leq n_{\mathcal{X}}}$.
• Figure 4: Qualitative comparisons of the regularization effect when adding a loss term on the normal map. On the bottom left is a generation mid-interpolation for a model trained only with the MSE loss, on the right is a similar generated mesh but with a model trained with a loss on the normal map.
• Figure 5: Qualitative comparisons of partial deformations on DFAUST, COMA and MANO. The As-Rigid-As-Possible deformations handle efficiently deformations that are close to linear (2nd and last row) but fail to adapt to large deviations.