Partial-to-Partial Shape Matching with Geometric Consistency

TL;DR

This work tackles partial-to-partial 3D shape matching by enforcing geometric consistency through a novel triangle-product, non-linear integer program that operates on product spaces of mesh elements. It integrates deep features with an exact optimization framework and introduces a pruning ILP solver together with a coarse-to-fine upsampling strategy to achieve practical runtimes. The authors also release PARTIALSMAL, an inter-class dataset derived from SMAL, and demonstrate state-of-the-art performance on both intra- and inter-class tasks using IoU and geodesic-error metrics, without requiring hole filling or labeled training data. While runtime remains a challenge and boundary cases pose limitations, the method provides a principled, data-efficient approach to partial-to-partial matching with strong geometric guarantees and broad applicability in 3D scanning and related fields.

Abstract

Finding correspondences between 3D shapes is an important and long-standing problem in computer vision, graphics and beyond. A prominent challenge are partial-to-partial shape matching settings, which occur when the shapes to match are only observed incompletely (e.g. from 3D scanning). Although partial-to-partial matching is a highly relevant setting in practice, it is rarely explored. Our work bridges the gap between existing (rather artificial) 3D full shape matching and partial-to-partial real-world settings by exploiting geometric consistency as a strong constraint. We demonstrate that it is indeed possible to solve this challenging problem in a variety of settings. For the first time, we achieve geometric consistency for partial-to-partial matching, which is realized by a novel integer non-linear program formalism building on triangle product spaces, along with a new pruning algorithm based on linear integer programming. Further, we generate a new inter-class dataset for partial-to-partial shape-matching. We show that our method outperforms current SOTA methods on both an established intra-class dataset and our novel inter-class dataset.

Figures (15)

• Figure 1: We present the first geometrically consistent partial-to-partial shape matching solution. Our approach can find correspondences between partial shapes and thus determine the overlapping region between these shapes (left). We can find correspondences within shape classes (middle), and across shape classes (right). For the latter, we introduce a new inter-class partial-to-partial matching dataset.
• Figure 2: Overview over our matching pipeline: The 3D meshes are fed into a feature extractor that returns vertex-wise features. From these vertex-wise features, we generate triangle-wise features to make them accessible for our algorithm. With these features, we define an integer program that ensures that every triangle is matched at most once for both shapes, that at least one non-boundary triangle is matched (per shape), and that neighboring relationships for inner triangles are fulfilled. We tackle this non-linear integer program by solving a subset of integer linear programs (ILPs) for a specific number of matchings.
• Figure 3: Illustration of our pruning algorithm: a) We first calculate an ILP for each initial value for $x$. Then, we identify intervals with potentially smaller $h(x)$ than the current minimum. b) We define new points in search space for $x$ and calculate ILPs for these points only. c) We continue till all necessary intervals are solved.
• Figure 4: Our upsampling strategy via nearest neighbor search: Given a matching between vertices $\mathbf{v}^{(\mathcal{X})}_i$ and $\mathbf{v}^{(\mathcal{Y})}_j$ on lower resolution, we determine the matchings in high resolution by only considering the upsampled vertices (dark blue). A vertex $\mathbf{v'}^{(\mathcal{X})}_i$, which was downsampled on $\mathbf{v}^{(\mathcal{X})}_i$ can only be matched on the upsampled vertices of $\mathbf{v}^{(\mathcal{Y})}_j$ (dark blue vertices in $\mathcal{Y}$). We choose this vertex $\mathbf{v'}^{(\mathcal{Y})}_j$ that has the smallest distance in feature space of $\mathbf{W}^{(\mathcal{X})}$ and $\mathbf{W}^{(\mathcal{Y})}$.
• Figure 5: Comparisons of overlapping region predictions on CP2P TEST and PARTIALSMAL. DPFM attaiki2021dpfm often returns noisy predictions and Sm-comb roetzer2022scalable predicts that the shape is matched completely or at the wrong position in the given example shapes (first two rows CP2P TEST, last two rows PARTIALSMAL). Only our method returns results close to the expected ground truth. We denote that our method performs overlapping prediction and matching simultaneously.