Isometric Multi-Shape Matching

TL;DR

The paper addresses the problem of extracting cycle-consistent isometric correspondences across a collection of 3D shapes by introducing a unified universe-based formulation that couples shape-to-universe permutations with shape-to-universe functional maps. It proposes IsoMuSh, an efficient alternating-projection algorithm that updates a universe-anchored matching matrix $U$ and a universe-aligned functional map $Q$, with projections onto partial-permutation and orthogonal constraint sets to guarantee cycle-consistency by construction. The approach achieves state-of-the-art performance on standard benchmarks (TOSCA, FAUST, SCAPE), including challenging partial-shape scenarios, while providing convergence guarantees and a detailed complexity analysis. This framework enables accurate texture transfer, as well as potential integration with deep learning to construct robust, scalable isometric shape collections for applications in 3D reconstruction and analysis.

Abstract

Finding correspondences between shapes is a fundamental problem in computer vision and graphics, which is relevant for many applications, including 3D reconstruction, object tracking, and style transfer. The vast majority of correspondence methods aim to find a solution between pairs of shapes, even if multiple instances of the same class are available. While isometries are often studied in shape correspondence problems, they have not been considered explicitly in the multi-matching setting. This paper closes this gap by proposing a novel optimisation formulation for isometric multi-shape matching. We present a suitable optimisation algorithm for solving our formulation and provide a convergence and complexity analysis. Our algorithm obtains multi-matchings that are by construction provably cycle-consistent. We demonstrate the superior performance of our method on various datasets and set the new state-of-the-art in isometric multi-shape matching.

Figures (10)

• Figure 1: Left: We present a novel approach for isometric multi-shape matching based on matching each shape to a (virtual) universe shape (shown semi-transparent). Our formulation represents point-to-point correspondences between shapes $i$ and $j$ as the composition of the shape-to-universe permutation matrix $P_i$ and the universe-to-shape permutation matrix $P_j^T$. By doing so, the pairwise matchings $P_{ij} = P_iP_j^\top$ are by construction cycle-consistent. Middle: Our formulation successfully solves isometric multi-matching of partial shapes. Right: Due to the cycle-consistency we can use our correspondences to faithfully transfer textures across a shape collection.
• Figure 2: The method by Cosmo et al. cosmo2017consistent leads to extremely sparse multi-matchings (middle), whereas our method obtains dense matchings (right).
• Figure 3: Percentage of correct keypoints (PCK) curves for five methods on three datasets, TOSCA, FAUST and SCAPE. Our method leads to better PCK curves (also see the AUC in Tab. \ref{['table:resultSummary']}) than its competitors across all datasets. Dashed lines indicate methods that do not jointly optimise for multi-matchings.
• Figure 4: Qualitative examples of correspondences on the TOSCA dog class. Black indicates no matching due to non-bijectivity. Our method is cycle-consistent and improves upon the non-smooth and noisy correspondences of the two-stage initialisation obtained via ZoomOut+Sync, whereas HiPPI does not (red circles). ZoomOut and ConsistentZoomOut have many unmatched points (black areas). $^\ddagger$ConsistentZoomOut obtains cycle-consistent $\mathcal{C}_{ij}$, but not $P_{ij}$. (Best viewed magnified on screen)
• Figure 5: Qualitative examples of correspondences on SCAPE. Black indicates no matching due to non-bijectivity. As in Fig. \ref{['fig:toscaQual']}, our results contain the least noise and are cycle-consistent, although there is one outlier shape where neither HiPPI nor our method could recover from a bad initialisation. $^\ddagger$ConsistentZoomOut obtains cycle-consistent $\mathcal{C}_{ij}$, but not $P_{ij}$. (Best viewed magnified on screen)

Theorems & Definitions (10)

• Lemma 1
• Proposition 2: Monotonicity of $U$-update
• Lemma 3
• Proposition 4: Monotonicity of $Q$-update
• Theorem 5: Convergence
• Lemma 6
• Proposition 7: Monotonicity of $U$-update
• Lemma 8
• Proposition 9: Monotonicity of $Q$-update
• Theorem 10: Convergence