CoE: Deep Coupled Embedding for Non-Rigid Point Cloud Correspondences

TL;DR

This work proposes a new shape representation -- a per-point high dimensional embedding, in an embedding space where semantically similar points share similar embeddings, that is aware of the underlying shape geometry and robust to shape deformations and various shape artefacts, such as noise and partiality.

Abstract

The interest in matching non-rigidly deformed shapes represented as raw point clouds is rising due to the proliferation of low-cost 3D sensors. Yet, the task is challenging since point clouds are irregular and there is a lack of intrinsic shape information. We propose to tackle these challenges by learning a new shape representation -- a per-point high dimensional embedding, in an embedding space where semantically similar points share similar embeddings. The learned embedding has multiple beneficial properties: it is aware of the underlying shape geometry and is robust to shape deformations and various shape artefacts, such as noise and partiality. Consequently, this embedding can be directly employed to retrieve high-quality dense correspondences through a simple nearest neighbor search in the embedding space. Extensive experiments demonstrate new state-of-the-art results and robustness in numerous challenging non-rigid shape matching benchmarks and show its great potential in other shape analysis tasks, such as segmentation.

Figures (16)

• Figure 1: Examples of LBO eigenbases and our learned coupled embeddings on a pair of non-rigidly deformed shapes. Ours are consistent while LBO eigenbases suffer from sign flip (cf. Fig. \ref{['fig:supp_basis_m']} for more examples).
• Figure 2: Pipeline overview. Given a pair of shapes ${\mathcal{S}}$ and ${\mathcal{T}}$ represented in point clouds, Our embedding extractor -- ASAP DiffusionNet with shared weights $\theta$ (not to be confused with generative diffusion models ho2020ddpmsong2021scorebased), extracts the intermediate per-point embeddings $\hat{\mathbf{\Psi}}_{\mathcal{S}}$ and $\hat{\mathbf{\Psi}}_{\mathcal{T}}$, which are further refined by the subsequent cross attention block to output the final coupled embeddings $\mathbf{\Psi}_{\mathcal{S}}$ and $\mathbf{\Psi}_{\mathcal{T}}$. The cross attention block constructs a complete bipartite graph that connects every point on the shape ${\mathcal{S}}$ with every points on the shape ${\mathcal{T}}$ to enable their cross-communication. Our unsupervised loss encourages the predicted embeddings of both shapes to be coupled while closely resembling the LBO eigenbases.
• Figure 3: Qualitative result on DT4D-M. Ours produces the most accurate and smooth correspondences, despite highly non-isometric deformation (errors highlighted in red).
• Figure 4: Generalisation from the training set SURREAL to the test set SHREC19. Our method generalises better compared to baselines (errors highlighted in red).
• Figure 5: Robustness against topological changes (the left shoulder and face of the kid are glued together). Ours is least sensitive to this noise among all competing methods (errors highlighted in red).