Combinative Matching for Geometric Shape Assembly

TL;DR

This work addresses geometric shape assembly by introducing combinative matching, which jointly models identical surface shape and opposite volume occupancy to resolve local matching ambiguities. The method uses rotation-equivariant feature embeddings, separate shape and occupancy descriptors, and an optimal-transport–based correspondence step to estimate robust inter-part transforms via weighted SVD. Training combines orientation, shape, occupancy, and point-matching losses, yielding a cost matrix that rewards both surface similarity and volumetric complementarity. Experimental results on Breaking Bad show state-of-the-art performance for pairwise and multi-part assembly, with strong generalization across object domains, and ablations confirm the necessity of occupancy modeling and equivariant representations for reliable interlocking.

Abstract

This paper introduces a new shape-matching methodology, combinative matching, to combine interlocking parts for geometric shape assembly. Previous methods for geometric assembly typically rely on aligning parts by finding identical surfaces between the parts as in conventional shape matching and registration. In contrast, we explicitly model two distinct properties of interlocking shapes: 'identical surface shape' and 'opposite volume occupancy.' Our method thus learns to establish correspondences across regions where their surface shapes appear identical but their volumes occupy the inverted space to each other. To facilitate this process, we also learn to align regions in rotation by estimating their shape orientations via equivariant neural networks. The proposed approach significantly reduces local ambiguities in matching and allows a robust combination of parts in assembly. Experimental results on geometric assembly benchmarks demonstrate the efficacy of our method, consistently outperforming the state of the art. Project page: https://nahyuklee.github.io/cmnet.

Figures (19)

• Figure 1: Combinative matching. In contrast to conventional approaches to matching solely based on shape similarity, our combinative matching explicitly models two distinct properties of interlocking shapes, 'identical surface shape' and 'opposite volume occupancy,' and learns to establish correspondences across regions where their surface shapes appear identical but their volumes occupy the inverted space to each other. The figure shows the assembly of source (gray) and target (red & blue) parts, with a true match shown by green dots ($\bullet$) connected by line. The color gradient on target points indicates correlation scores with the green source point, ranging from red (high) to blue (low). Incorporating the volume occupancy (shown in this example), reduces visual ambiguities, achieving accurate assembly.
• Figure 2: Main concept of our combinative matching.
• Figure 3: Overall architecture. Here, we show core components of (a) feature embedding network, (b) surface shape matching branch, (c) volume occupancy matching branch, and (d) transformation estimation. We refer the readers to Sec. \ref{['sec:network']} for details of each component.
• Figure 4: Visualization of learned orientations. We visualize learned vectors of $\{\mathbf{x}_i\}_{i \in \mathcal{I}}$ (left, red arrows) and $\{\mathbf{y}_i\}_{i \in \mathcal{I}}$ (middle, green arrows). The assembly result is shown on the right.
• Figure 5: Visualization of correlation distribution. A green dot ($\bullet$) on the left point cloud marks the source's $i$-th point, with corresponding true match points marked with green dots and arrows. Point colors represent correlation score magnitudes for the $i$-th point's similarity to each target point, with red and blue indicating high and low correlation scores, respectively.