Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural varifolds: an aggregate representation for quantifying the geometry of point clouds

Juheon Lee, Xiaohao Cai, Carola-Bibian Schönlieb, Simon Masnou

TL;DR

This work introduces neural varifolds, a geometry-aware representation for point clouds that encodes surfaces as measures over positions and tangent planes, enabling principled comparison via RKHS-based kernels. By leveraging neural tangent kernels, the authors build two variants (NTK1 and NTK2) that yield varifold-based similarity metrics across sets of point clouds, and demonstrate their utility for shape matching, few-shot shape classification, and 3D shape reconstruction. The approach achieves strong performance in shape matching and few-shot classification, and competes with state-of-the-art reconstruction methods, while presenting favorable computation in several settings. The neural varifold framework advances geometry-aware learning on unstructured 3D data by connecting deep learning with geometric measure theory and NTK theory.

Abstract

Point clouds are popular 3D representations for real-life objects (such as in LiDAR and Kinect) due to their detailed and compact representation of surface-based geometry. Recent approaches characterise the geometry of point clouds by bringing deep learning based techniques together with geometric fidelity metrics such as optimal transportation costs (e.g., Chamfer and Wasserstein metrics). In this paper, we propose a new surface geometry characterisation within this realm, namely a neural varifold representation of point clouds. Here the surface is represented as a measure/distribution over both point positions and tangent spaces of point clouds. The varifold representation quantifies not only the surface geometry of point clouds through the manifold-based discrimination, but also subtle geometric consistencies on the surface due to the combined product space. This study proposes neural varifold algorithms to compute the varifold norm between two point clouds using neural networks on point clouds and their neural tangent kernel representations. The proposed neural varifold is evaluated on three different sought-after tasks -- shape matching, few-shot shape classification and shape reconstruction. Detailed evaluation and comparison to the state-of-the-art methods demonstrate that the proposed versatile neural varifold is superior in shape matching and few-shot shape classification, and is competitive for shape reconstruction.

Neural varifolds: an aggregate representation for quantifying the geometry of point clouds

TL;DR

This work introduces neural varifolds, a geometry-aware representation for point clouds that encodes surfaces as measures over positions and tangent planes, enabling principled comparison via RKHS-based kernels. By leveraging neural tangent kernels, the authors build two variants (NTK1 and NTK2) that yield varifold-based similarity metrics across sets of point clouds, and demonstrate their utility for shape matching, few-shot shape classification, and 3D shape reconstruction. The approach achieves strong performance in shape matching and few-shot classification, and competes with state-of-the-art reconstruction methods, while presenting favorable computation in several settings. The neural varifold framework advances geometry-aware learning on unstructured 3D data by connecting deep learning with geometric measure theory and NTK theory.

Abstract

Point clouds are popular 3D representations for real-life objects (such as in LiDAR and Kinect) due to their detailed and compact representation of surface-based geometry. Recent approaches characterise the geometry of point clouds by bringing deep learning based techniques together with geometric fidelity metrics such as optimal transportation costs (e.g., Chamfer and Wasserstein metrics). In this paper, we propose a new surface geometry characterisation within this realm, namely a neural varifold representation of point clouds. Here the surface is represented as a measure/distribution over both point positions and tangent spaces of point clouds. The varifold representation quantifies not only the surface geometry of point clouds through the manifold-based discrimination, but also subtle geometric consistencies on the surface due to the combined product space. This study proposes neural varifold algorithms to compute the varifold norm between two point clouds using neural networks on point clouds and their neural tangent kernel representations. The proposed neural varifold is evaluated on three different sought-after tasks -- shape matching, few-shot shape classification and shape reconstruction. Detailed evaluation and comparison to the state-of-the-art methods demonstrate that the proposed versatile neural varifold is superior in shape matching and few-shot shape classification, and is competitive for shape reconstruction.
Paper Structure (28 sections, 2 theorems, 18 equations, 7 figures, 10 tables)

This paper contains 28 sections, 2 theorems, 18 equations, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Our contributions:
  3. Related works
  4. Geometric deep learning on point clouds.
  5. Varifolds.
  6. Varifold representations for point clouds
  7. Neural tangent kernel.
  8. Neural varifold computation
  9. Experiments
  10. Dataset and experimental setting.
  11. Shape matching
  12. Few-shot shape classification
  13. Shape reconstruction
  14. Limitations
  15. Appendix
  16. ...and 13 more sections

Key Result

Proposition 3.3

hsieh2019metrics. Let $k_{\rm pos}$ and $k_{G}$ be continuous positive definite kernels on $\mathbb{R}^n$ and $\tilde{G}(d,n)$, respectively. Assume in addition that for any $x \in \mathbb{R}^n$, $k_{\rm pos}(x,\cdot) \in C_0(\mathbb{R}^n)$. Then $k_{\rm pos} \otimes k_G$ is a positive definite kern

Figures (7)

  • Figure 1: Shape matching examples with different shape similarity metrics, i.e., CD, EMD, CT, NTK1 and NTK2. Hippo is a shortened term referring to the hippocampus.
  • Figure 2: Examples of the shape reconstruction comparison.
  • Figure 3: Visualisation of shape reconstruction results from SIREN, Neural Splines, NKSR and NTK1 for the Airplane, Bench and Cabinet categories.
  • Figure 4: Visualisation of shape reconstruction results from SIREN, Neural Splines, NKSR and NTK1 for the Car, Chair and Display categories.
  • Figure 5: Visualisation of shape reconstruction results from SIREN, Neural Splines, NKSR and NTK1 for the Lamp, Speaker and Rifle categories.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Definition 3.1: Rectifiable oriented $d$-varifolds
  • Definition 3.2: Bounded Lipschitz distance
  • Proposition 3.3
  • Corollary 3.4