Deep Frequency-Aware Functional Maps for Robust Shape Matching

TL;DR

This work proposes a novel unsupervised learning-based framework called Deep Frequency-Aware Functional Maps, which can gracefully cope with various shape matching scenarios and outperforms the existing state-of-the-art methods, especially in challenging settings like datasets with non-isometric deformation and inconsistent topology.

Abstract

Deep functional map frameworks are widely employed for 3D shape matching. However, most existing deep functional map methods cannot adaptively capture important frequency information for functional map estimation in specific matching scenarios, i.e., lacking \textit{frequency awareness}, resulting in poor performance when dealing with large deformable shape matching. To this end, we propose a novel unsupervised learning-based framework called Deep Frequency-Aware Functional Maps, which can gracefully cope with various shape matching scenarios. We first introduce a general constraint called Spectral Filter Operator Preservation to compute desirable functional maps, where the spectral filter operator encodes informative frequency information and can promote frequency awareness for deep functional map frameworks by learning a set of filter functions. Then, we directly utilize the proposed constraint as a loss function to supervise functional maps, pointwise maps, and filter functions simultaneously, where the filter functions are derived from the orthonormal Jacobi basis, and the coefficients of the basis are learnable parameters. Finally, we develop an effective refinement strategy to improve the final pointwise map, which incorporates our constraint and learned filter functions, leading to more robust and accurate correspondences during the inference process. Extensive experimental results on various datasets demonstrate that our approach outperforms the existing state-of-the-art methods, especially in challenging settings like datasets with non-isometric deformation and inconsistent topology.

Figures (5)

• Figure 1: We propose a novel unsupervised spectral shape matching approach that is more robust than UnsupRSFMNet Cao2023 across a broad range of challenging settings: shape matching with anisotropic meshing, shape matching of non-isometric shape pairs, shape matching with topological noise.
• Figure 2: The filter functions of different methods. (a) The filter functions in heat kernel preservation are low-pass, focusing on low frequency information only. (b) The filter functions of ZoomOut are upsampled ideal filters, using to upsample resolutions of eigenfunctions (c) Meyer wavelet functions consist of a low-pass and a set of band-pass in wavelet preservation are tight frames strictly (d) Our learned filter functions are optimized on non-isometric datasets SMAL_r.
• Figure 3: An overview of Deep FAFM. (1) Inputing a pair of shapes $\mathcal{M}$ and $\mathcal{N}$ to a trainable Siamese feature network to produce learned features $\mathbf{D}_{\mathcal{M}}$ and $\mathbf{D}_{\mathcal{N}}$ respectively. (2) Employing the filter learning layer to produce learned filter functions $H(\Lambda_{\mathcal{M}})$ and $H(\Lambda_{\mathcal{N}})$ respectively, i.e., Eq. \ref{['eq: Jacobi combination']}. (3) Utilizing learned features to compute the differentiable pointwise map $\Pi_{\mathcal{MN}}$ and functional map $C_{\mathcal{NM}}$ by resorting to Softmax operator (Eq. \ref{['eq: compute soft map']}) and functional map solver (Eq. \ref{['equ: desc and reg']}) respectively. (4) A frequency awareness loss term (Eq. \ref{['equ: frequency couple loss']}) is used to supervise the functional map $C_{\mathcal{NM}}$, pointwise map $\Pi_{\mathcal{MN}}$, as well as $H(\Lambda_{\mathcal{M}})$ and $H(\Lambda_{\mathcal{N}})$. (5) Using our refinement technique (Eq. \ref{['MCFP compute C']}) to refine a pointwise map $\Pi_{\mathcal{MN}}$, resulting in a more accurate and robust final correspondence $\Pi^{final}_{\mathcal{MN}}$ at the test stage.
• Figure 4: Comparisons with other methods on non-isometric shape matching. Correspondence is visualized by color transfer for shapes from the SMAL_r zuffi20173d (e.g. first row) and the DT4D-H datasets Magnet2022 (e.g. second row). Less error and color distortion happened in our method compared with the others, which shows our approach demonstrates superior matching performance for non-isometric shapes.
• Figure 5: Comparisons with other methods on shape matching with topological noise, where shapes from SHREC’16 TOPKIDS benchmark lahner2016shrec. The smoother and more accurate color distribution of our result illustrates our approach is more robust to topological noise compared to existing methods.

Theorems & Definitions (6)

• Remark 4.1
• proof
• Remark 4.2
• proof
• Remark A.1
• proof