NeuralSSD: A Neural Solver for Signed Distance Surface Reconstruction

TL;DR

This work tackles the challenge of reconstructing accurate, watertight 3D surfaces from sparse and noisy point clouds by introducing NeuralSSD, a hybrid framework that couples a structure-aware sparse convolution predictor with a differentiable closed-form SDF solver via a neural Galerkin formulation. The method builds a spatially varying basis expansion on a multi-scale sparse voxel grid and solves for basis coefficients through a variational energy that jointly enforces data fidelity, Hessian-based smoothness, and point constraints, yielding robust and detailed reconstructions. Key contributions include a novel energy formulation $E(f)=E_N(f)+\lambda_1E_H(f)+\lambda_2E_P(f)$, an adaptive sparse network predicting normals and bases, a closed-form solver for $\boldsymbol{\alpha}$, and a Point-Voxel Attention mechanism that preserves intra-voxel geometry; the approach achieves state-of-the-art accuracy and generalization on ShapeNet and Matterport3D while maintaining competitive efficiency. The practical impact lies in scalable, high-fidelity 3D reconstruction for robotics, graphics, and AR/VR, with end-to-end differentiability enabling learning-based priors without sacrificing geometric fidelity.

Abstract

We proposed a generalized method, NeuralSSD, for reconstructing a 3D implicit surface from the widely-available point cloud data. NeuralSSD is a solver-based on the neural Galerkin method, aimed at reconstructing higher-quality and accurate surfaces from input point clouds. Implicit method is preferred due to its ability to accurately represent shapes and its robustness in handling topological changes. However, existing parameterizations of implicit fields lack explicit mechanisms to ensure a tight fit between the surface and input data. To address this, we propose a novel energy equation that balances the reliability of point cloud information. Additionally, we introduce a new convolutional network that learns three-dimensional information to achieve superior optimization results. This approach ensures that the reconstructed surface closely adheres to the raw input points and infers valuable inductive biases from point clouds, resulting in a highly accurate and stable surface reconstruction. NeuralSSD is evaluated on a variety of challenging datasets, including the ShapeNet and Matterport datasets, and achieves state-of-the-art results in terms of both surface reconstruction accuracy and generalizability.

Figures (5)

• Figure 1: The NeuralSSD pipeline. Given a raw point cloud, we first construct a multi-scale sparse voxel grid. An adaptive sparse network then predicts spatially varying basis functions and normals within this grid. Finally, a closed-form, differentiable SDF solver computes the basis coefficients to reconstruct a watertight surface. This hybrid approach combines a learned, structure-aware prior with a variational optimization that explicitly enforces data fidelity, enabling robust and detailed reconstruction from sparse and noisy inputs.
• Figure 2: When focusing on the two-dimensional special case for ease of understanding, the original multi-layer convolutional kernel consists of 29 elements, derived from layers of different sizes from three neighboring hierarchies, as illustrated in \ref{['origin kernel']}. During the convolution operation, the voxels within the convolution kernel are filtered based on the number of points computed inside the voxel, as shown in \ref{['filtered kernel']}. This results in a convolutional kernel of the form \ref{['multi-layer kernel']}. The actual convolution process, however, occurs in three dimensions, where the original multi-layer convolutional kernel consists of 107 voxels, originating from three adjacent hierarchies.
• Figure 3: Here are the results on the ShapeNet datasetchangshapenet across three different settings. Methods SPSR utilize points normals, while others do not. Notably, our method demonstrates a capability to accurately reconstruct fine geometric details (such as the car wheels).
• Figure 4: Qualitative results on the Matterport3D dataset. Our method demonstrates superior performance in reconstructing large-scale indoor scenes, excelling at capturing fine details, such as the chair highlighted in the red box, while simultaneously ensuring overall structural integrity by eliminating noise from surfaces like walls and floors.
• Figure 5: Qualitative comparison on the Surface Reconstruction Benchmark (SRB). Our method achieves a better balance between surface smoothness and detail preservation. The error is visualized with a color map where blue indicates low error and red indicates high error.