DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features

TL;DR

DV-Matcher tackles dense non-rigid point cloud matching without relying on meshes or ground-truth correspondences. It integrates a visual encoding stage that brings semantic cues from pre-trained vision models with a deformation-based proxy loss (alongside ARAP, smoothness, and geodesic-consistency terms) to guide feature learning and correspondence inference directly on point clouds. The approach achieves state-of-the-art results across near-isometric, heterogeneous, partial, and real-world data, including medical scans and garment scans, demonstrating strong generalization and robustness. This framework enables practical, label-free, preprocessing-free matching with potential applications in 3D reconstruction, texture transfer, and cross-domain analysis, though it incurs non-real-time runtime and leaves room for improved orientation robustness.

Abstract

In this paper, we present DV-Matcher, a novel learning-based framework for estimating dense correspondences between non-rigidly deformable point clouds. Learning directly from unstructured point clouds without meshing or manual labelling, our framework delivers high-quality dense correspondences, which is of significant practical utility in point cloud processing. Our key contributions are two-fold: First, we propose a scheme to inject prior knowledge from pre-trained vision models into geometric feature learning, which effectively complements the local nature of geometric features with global and semantic information; Second, we propose a novel deformation-based module to promote the extrinsic alignment induced by the learned correspondences, which effectively enhances the feature learning. Experimental results show that our method achieves state-of-the-art results in matching non-rigid point clouds in both near-isometric and heterogeneous shape collection as well as more realistic partial and noisy data.

Figures (21)

• Figure 1: We train DV-Matcher and two baselines under a challenging setting. The training set consists of only one full point cloud as reference and a set of $516$ partial point clouds sampled from shapes in SHREC'19 with significant pose/style deformations (see the yellow point clouds). Without any correspondence label, our framework not only manages to match accurately on SHREC'19 benchmark (both partial and full setting), but also generalizes well to unseen benchmarks. See Sec. \ref{['sec:realapp']} for more details.
• Figure 2: The schematic illustration of our pipeline.
• Figure 3: Illustration of our deformer, which predicts rigid transformation at each deformation graph node.
• Figure 4: Ablation study on our training losses.
• Figure 5: Qualitative results of SCAPE-PV and noisy real scans.