Deformation Recovery: Localized Learning for Detail-Preserving Deformations

TL;DR

This work tackles the problem of producing high-quality, detail-preserving 3D shape deformations without relying on global shape encodings. It introduces the Local Jacobian Network (LJN), a set of per-face MLPs that, given a coarse local deformation signal, predict detailed Jacobians $J_{12}$ which are then integrated via a Poisson solve to recover embeddings. By operating on local, shared-weight representations and leveraging spectral projections, LJN achieves strong cross-category generalization, enabling accurate map refinement, unsupervised deformation and mapping, and interactive editing with significantly faster inference than iterative baselines. The approach supports meshes with differing connectivity through functional maps, and it demonstrates data efficiency (training on ~60 shapes) and competitive performance on standard benchmarks, including FAUST-Challenge. Limitations include potential volume shrinkage and handling of sharp bends, with future work pointing toward physics-informed energies and alternative deformation spaces such as the Discrete Shell-Operator.

Abstract

We introduce a novel data-driven approach aimed at designing high-quality shape deformations based on a coarse localized input signal. Unlike previous data-driven methods that require a global shape encoding, we observe that detail-preserving deformations can be estimated reliably without any global context in certain scenarios. Building on this intuition, we leverage Jacobians defined in a one-ring neighborhood as a coarse representation of the deformation. Using this as the input to our neural network, we apply a series of MLPs combined with feature smoothing to learn the Jacobian corresponding to the detail-preserving deformation, from which the embedding is recovered by the standard Poisson solve. Crucially, by removing the dependence on a global encoding, every \textit{point} becomes a training example, making the supervision particularly lightweight. Moreover, when trained on a class of shapes, our approach demonstrates remarkable generalization across different object categories. Equipped with this novel network, we explore three main tasks: refining an approximate shape correspondence, unsupervised deformation and mapping, and shape editing. Our code is made available at https://github.com/sentient07/LJN

Figures (18)

• Figure 1: Illustration of our learning framework with spectrally projected input signal. Given a source shape and the coarse deformation signal, we feed in individual triangle deformations, as Jacobians, averaged to incident vertices. Then, we apply series MLPs coupled with spectral projection in the feature-space to recover detail-preserving deformation.
• Figure 2: Illustration of the fidelity of our training framework. We select a seed vertex, marked with a dotted circle. The Frobenius norm of the difference between the Jacobian at this seed vertex and all other vertices is plotted. The similarity in the distribution between the input (projected Jacobian) and the ground truth suggests the well-posed nature of our learning problem.
• Figure 3: Visual insights on how our input signal is affected by near-isometric and non-isometric deformations. In the first row, we show a human shape undergoing two deformations into a similar pose but taken by two subjects, one similar and one not. We compare the discrepancy between the input Jacobian and the ground truth Jacobian as the Frobenius norm (plotted in the second and third columns). We repeat this for a pair of animals with a significantly higher degree of non-isometry between them. We observe that our input signal $\Theta_{12}$, remains comparable to the ground-truth across different levels of non-isometry.
• Figure 4: Given a source (a) and a deformation (Jacobian), we compute the embedding following Eqn \ref{['eqn:Poisson']} and visualize in (b). While the deformation is near-perfect, the surfaces do not 'align' as shown in (c) where (b) is juxtaposed to the target shape. Computing the embedding via Eqn \ref{['eqn:EmbedRecoveryBacksub']} yields a surface that is geometrically closer to the target as shown in (d).
• Figure 5: Illustrating how LJN achieves high map-refinement accuracy: Given (a) the source shape and (b) an input map containing high-frequency artifacts, (c) we convert it to the Functional Map and pull the coordinates. The pulled coordinates are devoid of high-frequency artifacts. Subsequently, (d) our reconstruction and (e) the obtained map are detail-preserving. (f) denotes the target shape. We color-code correspondence from refined maps.