Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Descriptors: Self-Supervised Learning of Robust Local Surface Descriptors Using Polynomial Patches

Gal Yona, Roy Velich, Ron Kimmel, Ehud Rivlin

TL;DR

This work tackles robust, dense shape correspondence under non-rigid deformations, partial observations, and topological noise by learning sampling-invariant local descriptors through self-supervision. It introduces a synthetic data pipeline of local patches defined by random polynomials and a DeltaConv-based feature extractor trained with SimSiam to produce $2048$-dimensional patch descriptors, with per-point features of $512$ dimensions. The approach achieves state-of-the-art results on FAUST, SCAPE, TOPKIDS, and SHREC'16, demonstrating superior robustness to holes and topological perturbations and strong integration with existing shape-matching pipelines such as RobustFMNet. This yields practical impact by enabling more reliable 3D shape analysis in scenes with incomplete or noisy data, while highlighting areas for future improvement in anisotropic meshes.

Abstract

Classical shape descriptors such as Heat Kernel Signature (HKS), Wave Kernel Signature (WKS), and Signature of Histograms of OrienTations (SHOT), while widely used in shape analysis, exhibit sensitivity to mesh connectivity, sampling patterns, and topological noise. While differential geometry offers a promising alternative through its theory of differential invariants, which are theoretically guaranteed to be robust shape descriptors, the computation of these invariants on discrete meshes often leads to unstable numerical approximations, limiting their practical utility. We present a self-supervised learning approach for extracting geometric features from 3D surfaces. Our method combines synthetic data generation with a neural architecture designed to learn sampling-invariant features. By integrating our features into existing shape correspondence frameworks, we demonstrate improved performance on standard benchmarks including FAUST, SCAPE, TOPKIDS, and SHREC'16, showing particular robustness to topological noise and partial shapes.

Neural Descriptors: Self-Supervised Learning of Robust Local Surface Descriptors Using Polynomial Patches

TL;DR

This work tackles robust, dense shape correspondence under non-rigid deformations, partial observations, and topological noise by learning sampling-invariant local descriptors through self-supervision. It introduces a synthetic data pipeline of local patches defined by random polynomials and a DeltaConv-based feature extractor trained with SimSiam to produce -dimensional patch descriptors, with per-point features of dimensions. The approach achieves state-of-the-art results on FAUST, SCAPE, TOPKIDS, and SHREC'16, demonstrating superior robustness to holes and topological perturbations and strong integration with existing shape-matching pipelines such as RobustFMNet. This yields practical impact by enabling more reliable 3D shape analysis in scenes with incomplete or noisy data, while highlighting areas for future improvement in anisotropic meshes.

Abstract

Classical shape descriptors such as Heat Kernel Signature (HKS), Wave Kernel Signature (WKS), and Signature of Histograms of OrienTations (SHOT), while widely used in shape analysis, exhibit sensitivity to mesh connectivity, sampling patterns, and topological noise. While differential geometry offers a promising alternative through its theory of differential invariants, which are theoretically guaranteed to be robust shape descriptors, the computation of these invariants on discrete meshes often leads to unstable numerical approximations, limiting their practical utility. We present a self-supervised learning approach for extracting geometric features from 3D surfaces. Our method combines synthetic data generation with a neural architecture designed to learn sampling-invariant features. By integrating our features into existing shape correspondence frameworks, we demonstrate improved performance on standard benchmarks including FAUST, SCAPE, TOPKIDS, and SHREC'16, showing particular robustness to topological noise and partial shapes.

Paper Structure

This paper contains 18 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classical Geometric Descriptors.
  4. Differential Invariants and Signatures.
  5. Spectral Methods and Learning on 3D Shapes.
  6. Method
  7. Synthetic Data Generation
  8. Network Architecture
  9. Self-Supervised Training with SimSiam
  10. Experiments
  11. Qualitative Analysis of Feature Space
  12. Neural Descriptors and Axiomatic Shape Correspondence
  13. Neural Descriptors and Learning-Based Shape Correspondence
  14. Evaluation on Isometric Shape Collections.
  15. Robustness to Topological Noise.
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Architecture overview.
  • Figure 2: PCA visualization of feature spaces.Top Row: Our features maintain distinct clusters despite noise, while classical descriptors show dispersed, overlapping clusters. Bottom Row: Our features capture smooth geometric transitions with a well-ordered path, while traditional descriptors (HKS, WKS, and SHOT) produce discontinuous trajectories. Our visualization approach is inspired by the feature space representation techniques demonstrated in rotstein2024pathways.
  • Figure 3: Correspondence accuracy comparison on SHREC'16 partial shapes. Our method (purple) achieves better geodesic error rates compared to traditional descriptors.
  • Figure 4: Texture transfer on partial shapes. Follow the blue arrows for visual comparison. Zoom in for details.
  • Figure 5: Texture transfer in presence of topological noise. Follow the blue arrows for visual comparison. Zoom in for details.