NFR: Neural Feature-Guided Non-Rigid Shape Registration

TL;DR

This work introduces Neural Feature-Guided Registration (NFR), a learning-guided, correspondence-free framework for non-rigid 3D shape registration that robustly handles large deformations and partial inputs. By coupling a neural feature extractor trained via a deep functional maps teacher-student scheme with a geometric, two-stage registration process, NFR updates correspondences dynamically and enforces consistency to improve alignment in both ambient and learned feature spaces. A partial-DFR extension provides self-supervised training and spectral-embedding-based supervision to tackle partial-to-full matches, supported by a theoretical analysis of partial spectral embeddings. Across extensive benchmarks, NFR achieves state-of-the-art performance on non-rigid and partial shape registration, demonstrating strong generalization, robustness to topological perturbations, and applicability to medical data, with a noted limitation being the iterative optimization time.

Abstract

In this paper, we propose a novel learning-based framework for 3D shape registration, which overcomes the challenges of significant non-rigid deformation and partiality undergoing among input shapes, and, remarkably, requires no correspondence annotation during training. Our key insight is to incorporate neural features learned by deep learning-based shape matching networks into an iterative, geometric shape registration pipeline. The advantage of our approach is two-fold -- On one hand, neural features provide more accurate and semantically meaningful correspondence estimation than spatial features (e.g., coordinates), which is critical in the presence of large non-rigid deformations; On the other hand, the correspondences are dynamically updated according to the intermediate registrations and filtered by consistency prior, which prominently robustify the overall pipeline. Empirical results show that, with as few as dozens of training shapes of limited variability, our pipeline achieves state-of-the-art results on several benchmarks of non-rigid point cloud matching and partial shape matching across varying settings, but also delivers high-quality correspondences between unseen challenging shape pairs that undergo both significant extrinsic and intrinsic deformations, in which case neither traditional registration methods nor intrinsic methods work.

Figures (9)

• Figure 1: Shape registration methods like NDP li2022non and AMM AMM estimate intermediate correspondences via extrinsic proximity, therefore suffering from large intrinsic deformations. In contrast, our method successfully deforms a FAUST template (the left-most mesh) to another individual of a different pose (the right-most full or partial point cloud).
• Figure 2: We estimate correspondences between heterogeneous shapes from SHREC'07 with four learning-based methods, all trained on the SCAPE_r dataset. Our method outperforms the competing methods by a large margin. Remarkably, our method manages to deform a SCAPE template shape to heterogeneous shapes, as indicated by the blue shapes.
• Figure 3: Matching different partial shapes to a common template shape (left-most). (a) DPFM attaiki2021dpfm trained on Ours-S&F dataset; (b) DPFM attaiki2021dpfm trained on CUTattaiki2021dpfm; (c) HCLV2S huang20 trained on large scale SURREALhuang20 dataset ; (d) Ours trained on Ours-S&F dataset. DPFM attaiki2021dpfm and HCVL2S huang20 fails to handle unseen partial shapes effectively. Remarkablely, DPFM significantly struggles with disconnected shapes. Our method outperforms the competing methods by a large margin and achieve the consist results across different partiality.
• Figure 4: The schematic illustration of our pipeline. ${\mathbf{A}}$ is a pre-trained orientation regressor for aligning input shapes. Then a pre-trained feature extractor ${\mathbf{F}}$ embeds them into a high-dimensional canonical space. During the iterative optimization procedure of registration, correspondences are dynamically updated according to learned features (Stage-I) and coordinates (Stage-II) of the intermediate shapes. See more details in the text.
• Figure 5: Illustration of the partial spectral embedding construction and partial to full functional map processing during training.

Theorems & Definitions (1)

• Proposition 1