Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence

TL;DR

Stable-SCore rethinks 3D shape correspondence by combining a 2D foundation-model-based 2D character correspondence with a Semantic Flow Guided Registration that uses Neural Jacobian Fields for stable, dense 3D alignment under non-isometric deformations. The method trains a 2D adapter to produce robust semantic flows from multi-view renders and supervises differentiable 3D registration with geometry and flow losses, achieving state-of-the-art performance on non-isometric benchmarks and enabling downstream tasks like re-topology and texture transfer. A key contribution is the Character in-the-Wild (CharW) benchmark, designed to stress non-isometric variations and realism, helping drive progress in practical character correspondence. The work demonstrates that leveraging 2D foundation-model features for 2D-to-3D supervision, coupled with a stable deformation model and careful regularization, yields robust 3D correspondences that generalize across domains and support a range of applications.

Abstract

Establishing character shape correspondence is a critical and fundamental task in computer vision and graphics, with diverse applications including re-topology, attribute transfer, and shape interpolation. Current dominant functional map methods, while effective in controlled scenarios, struggle in real situations with more complex challenges such as non-isometric shape discrepancies. In response, we revisit registration-for-correspondence methods and tap their potential for more stable shape correspondence estimation. To overcome their common issues including unstable deformations and the necessity for careful pre-alignment or high-quality initial 3D correspondences, we introduce Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence. We first re-purpose a foundation model for 2D character correspondence that ensures reliable and stable 2D mappings. Crucially, we propose a novel Semantic Flow Guided Registration approach that leverages 2D correspondence to guide mesh deformations. Our framework significantly surpasses existing methods in challenging scenarios, and brings possibilities for a wide array of real applications, as demonstrated in our results.

Figures (11)

• Figure 1: We propose Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence. Given a source and a target mesh, our approach registers the source mesh to the target and establishes dense correspondence between them, with strong robustness to large variations in mesh topology, shape, and pose, as shown in (a). Our method enables a range of downstream applications, including re-topology, texture transfer, rig transfer, and shape interpolation, as shown in (b).
• Figure 2: Functional map methods face difficulties with non-isometric correspondences, while existing Registration-for-Correspondence methods break down without reliable initial correspondences. Our Stable-SCore maintains geometric fidelity and offers greater robustness to non-isometric shape variations in the wild.
• Figure 3: The Stable-SCore pipeline operates as follows: Source and target meshes are inputted and rendered into multi-view RGB or normal images using a fixed set of cameras. These images are processed through the network to extract 2D correspondences as a semantic flow map. Using differentiable rendering, we render forward flow under the same camera views and supervise it with the semantic flow. Chamfer Distance (CD) and normal loss, are also applied. The deformation model is iteratively optimized throughout this process.
• Figure 4: Visualized comparison with previous methods. All results are from the cross domain setting.
• Figure 5: Application of re-topology, texture transfer and shape interpolation.