Generalizing Shape-from-Template to Topological Changes

TL;DR

This work addresses monocular Shape-from-Template reconstruction when the object undergoes topological changes, such as tearing or incisions. It introduces a topological-change-aware SfT framework that starts from a classical SfT initialization and then adapts the template by partitioning its domain to accommodate tears, using tearing curves and four elementary classes. A depth-displacement field is optimized to correct depth near tear boundaries by minimizing an isometry-based cost, enabling consistent reconstructions across topological changes. The method is validated on synthetic and real-world datasets, showing improved accuracy over seven baselines and highlighting potential applications in surgical guidance, robotic cutting, and deformable-object tracking under topology changes.

Abstract

Reconstructing the surfaces of deformable objects from correspondences between a 3D template and a 2D image is well studied under Shape-from-Template (SfT) methods; however, existing approaches break down when topological changes accompany the deformation. We propose a principled extension of SfT that enables reconstruction in the presence of such changes. Our approach is initialized with a classical SfT solution and iteratively adapts the template by partitioning its spatial domain so as to minimize an energy functional that jointly encodes physical plausibility and reprojection consistency. We demonstrate that the method robustly captures a wide range of practically relevant topological events including tears and cuts on bounded 2D surfaces, thereby establishing the first general framework for topological-change-aware SfT. Experiments on both synthetic and real data confirm that our approach consistently outperforms baseline methods.

Figures (6)

• Figure 1: without topological changes bartoli2015shape. The known parametrized surface $\mathcal{T}$ with known parametrization $\Delta$ is deformed to produce the unknown deformed surface $\mathcal{S}.$ Using the known warp $\eta$ and the known projective image data $\mathcal{J},$ reconstructs the unknown depth function $\gamma$ which leads to the unknown parametrization $\varphi,$ as well as the unknown deformation $\Psi$.
• Figure 2: Schematic visualization of the four ETCs. In each case, the torn range $\Phi(\mathcal{M}\setminus\Gamma)$ is shown, representing isometric deformation of a flat sheet after tearing. The partial tears (a) and (b) leave the torn range connected, while cases (c) and (d) result in two connected components. In (d), the smaller, elevated component is homeomorphic to a disk, while the larger component is homeomorphic to an annulus.
• Figure 3: with topological changes. The formalism is as in \ref{['fig:SfT']}, but now the unknown is an isometry with topological change $\Phi$ with tearing curve $\Gamma.$ In particular, $\Phi$ is a simple disconnection .
• Figure 4: Dataset used in the experiments. (a)--(d): synthetic ; (e) and (f): torn papers.
• Figure 5: Reconstruction results from (a) owl data and (b) SAF data; the red hollow points are and the black dots are the reconstructions from our proposed method. The corresponding reconstructed and points are connected by think black lines.

Theorems & Definitions (6)

• definition 1
• definition 2
• definition 3
• Example 1
• definition 4
• Example 2