SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularization

Abstract

Establishing accurate point-to-point correspondences between non-rigid 3D shapes remains a critical challenge, particularly under non-isometric deformations and topological noise. Existing functional map pipelines suffer from ambiguities that geometric descriptors alone cannot resolve, and spatial inconsistencies inherent in the projection of truncated spectral bases to dense pointwise correspondences. In this paper, we introduce SGMatch, a learning-based framework for semantic-guided non-rigid shape matching. Specifically, we design a Semantic-Guided Local Cross-Attention module that integrates semantic features from vision foundation models into geometric descriptors while preserving local structural continuity. Furthermore, we introduce a regularization objective based on conditional flow matching, which supervises a time-varying velocity field to encourage spatial smoothness of the recovered correspondences. Experimental results on multiple benchmarks demonstrate that SGMatch achieves competitive performance across near-isometric settings and consistent improvements under non-isometric deformations and topological noise.

Figures (13)

• Figure 1: (Left): Colormap transfer on the SHREC'19 dataset demonstrates that incorporating semantic features resolves ambiguity and yields globally consistent correspondences. (Right): Vertex transfer on the SMAL dataset shows that the proposed conditional flow matching regularization promotes spatially smooth correspondences.
• Figure 2: Overview of SGMatch. Given a pair of shapes $\mathcal{X}$ and $\mathcal{Y}$, we extract geometric features $\mathbf{F}^{\mathrm{geo}}_{\mathcal{X}}, \mathbf{F}^{\mathrm{geo}}_{\mathcal{Y}}$ and semantic features $\mathbf{F}^{\mathrm{sem}}_{\mathcal{X}}, \mathbf{F}^{\mathrm{sem}}_{\mathcal{Y}}$, which are subsequently fused via the proposed SGLCA module. The resulting fused representations $\mathbf{F}^{\mathrm{fuse}}_{\mathcal{X}}$ and $\mathbf{F}^{\mathrm{fuse}}_{\mathcal{Y}}$ are then used to estimate functional maps $\mathbf{C}_{\mathcal{XY}}$ and to recover dense point-wise correspondences $\mathbf{\Pi}_{\mathcal{YX}}$. In parallel, spectral heat diffusion followed by conditional flow matching regularization constrains feature transport, thereby suppressing local mismatches and promoting locally smooth correspondences.
• Figure 3: Left: Near-isometric matching and cross-dataset generalisation on FAUST, SCAPE, and SHREC'19. Best results are highlighted. Right: Qualitative results on the challenging SHREC'19 dataset.
• Figure 4: PCK curves and AUC values. Left: Non-isometric matching on SMAL and DT4D-H. Right: Matching with topological noise on TOPKIDS. Our approach achieves strong performance in both settings, outperforming existing methods.
• Figure 5: Qualitative Results on SMAL and DT4D-H. Comparison of our method against DeepFAFM and HybridFMap, via texture transfer.