MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step

TL;DR

A novel method is proposed, named MultiPull, to learn multi-scale implicit fields from raw point clouds by optimizing accurate SDFs from coarse to fine by mapping 3D query points into a set of frequency features, which makes it possible to leverage multi-level features during optimization.

Abstract

Reconstructing a continuous surface from a raw 3D point cloud is a challenging task. Recent methods usually train neural networks to overfit on single point clouds to infer signed distance functions (SDFs). However, neural networks tend to smooth local details due to the lack of ground truth signed distances or normals, which limits the performance of overfitting-based methods in reconstruction tasks. To resolve this issue, we propose a novel method, named MultiPull, to learn multi-scale implicit fields from raw point clouds by optimizing accurate SDFs from coarse to fine. We achieve this by mapping 3D query points into a set of frequency features, which makes it possible to leverage multi-level features during optimization. Meanwhile, we introduce optimization constraints from the perspective of spatial distance and normal consistency, which play a key role in point cloud reconstruction based on multi-scale optimization strategies. Our experiments on widely used object and scene benchmarks demonstrate that our method outperforms the state-of-the-art methods in surface reconstruction.

Figures (16)

• Figure 1: Visualization of the 3D shape reconstruction. In (a), (b) and (c), SDFs are learned from a point cloud by optimizing multi-level query points at multi-step. At each step, we optimize query points at one level with frequency features at this specific level as conditions. This enables the network to progressively recover coarse-to-fine geometry details.
• Figure 2: Overview of our method: (a) Frequency Feature Transformation (FFT) module and (b) Multi-Step Pulling (MSP) module. In (a), we learn Fourier bases $h_{i}(Q)$ from query points $Q$ using the Fourier layer and obtain multi-level frequency features ${y_{i}}$ through Hadamard product. In (b), using multi-level frequency features from (a) and a linear network LSNN with shared parameters, we calculate the distance(D) of $Q_ {i}$ to its corresponding surface target point $Q_{t}$ to predict a more accurate surface. We visualize the predicted SDF distribution map corresponding to the frequency features in (a) and the reconstruction from each step of SDF predictions on the right side of (b).
• Figure 3: Comparison of parameter distributions of different linear layers especially in ($L_2,L_4,L_6,L_8$). We show the different initialization strategies on the results of the reconstruction task and the visualization effects in Appendix \ref{['sec:app_ab']}.
• Figure 4: Visual comparison of reconstructions on ShapeNet.
• Figure 5: Reconstruction accuracy on FAMOUS in terms of CD$_{L2}$ and NC.