A Survey and Benchmark of Automatic Surface Reconstruction from Point Clouds

TL;DR

The paper tackles the challenge of reconstructing surfaces from point clouds by benchmarking traditional and learning-based methods under realistic defects. It combines a broad literature survey with a standardized benchmark that uses synthetic and real data, enabling fair comparisons and generalization testing. Key findings show that learning-based methods excel when training data resembles test data, but struggle with unseen defects, while traditional methods are more robust across diverse acquisitions; hybrid approaches offer favorable trade-offs. The work provides practical insights for selecting reconstruction strategies and highlights the need for training data that captures real-world acquisition variability, guiding future research in robust surface reconstruction.

Abstract

We present a comprehensive survey and benchmark of both traditional and learning-based methods for surface reconstruction from point clouds. This task is particularly challenging for real-world acquisitions due to factors such as noise, outliers, non-uniform sampling, and missing data. Traditional approaches often simplify the problem by imposing handcrafted priors on either the input point clouds or the resulting surface, a process that can require tedious hyperparameter tuning. In contrast, deep learning models have the capability to directly learn the properties of input point clouds and desired surfaces from data. We study the influence of handcrafted and learned priors on the precision and robustness of surface reconstruction techniques. We evaluate various time-tested and contemporary methods in a standardized manner. When both trained and evaluated on point clouds with identical characteristics, the learning-based models consistently produce higher-quality surfaces compared to their traditional counterparts -- even in scenarios involving novel shape categories. However, traditional methods demonstrate greater resilience to the diverse anomalies commonly found in real-world 3D acquisitions. For the benefit of the research community, we make our code and datasets available, inviting further enhancements to learning-based surface reconstruction. This can be accessed at https://github.com/raphaelsulzer/dsr-benchmark .

Figures (8)

• Figure 1: Difficulties in surface reconstruction from point clouds: In each plot, we show the real surface , point samples , and possible reconstructions . The correct topology and geometry of the real surface are not known from the point samples (\ref{['fig:ch1:topology']},\ref{['fig:ch1:geometry']}). The point samples may also include acquisition defects such as noise (\ref{['fig:ch1:defects']}). The goal of any surface reconstruction algorithm is finding a good approximation of the real surface, in terms of its geometry and topology. Learning-based surface reconstruction can learn shape patterns or sampling errors such as the one exemplified here, and use the learned knowledge during reconstruction for a better approximation.
• Figure 2: Properties of surface meshes: In (\ref{['properties:a']}), we show a surface mesh that is non-watertight due to the hole in the center marked with red vertices and edges. The green vertices and edges mark the intersection with the domain boundary. In (\ref{['properties:b']}), we show a surface mesh with non-manifold edges and non-manifold vertices marked in red. In (\ref{['properties:c']}), we show a non-manifold and intersecting surface mesh. Here, the non-manifoldness is harder to detect, because it does not happen along edges of the mesh. In (\ref{['properties:d']}), we show a non-orientable surface.
• Figure 3: Survey overview: Surface reconstruction algorithms can be categorized based on their primary representation approach: surface-based or volume-based. Within each category, methods are further classified as either interpolation-based or approximation-based. Subsequently, specific subgroups and various individual methods are identified under each classification, providing a comprehensive framework for understanding the landscape of surface reconstruction techniques.
• Figure 4: Examples of surface reconstruction algorithms: In (\ref{['fig:survey:inter_surf']}), we exemplify a reconstruction from a surface-based interpolation approach. Close by points are connected with triangles to reconstruct the surface. In (\ref{['fig:survey:approx_surf']}), we show a reconstruction of a surface-based approximation approach. An initial mesh, e.g., a plane, is deformed to fit to the input points. In (\ref{['fig:survey:inter_vol']}), we show a volume-based interpolation. Input points are connected using a 3D Delaunay tetrahedralization. The resulting tetrahedra are then labelled as inside (dark blue) or outside the surface (red) and the interface triangles constitute the final mesh. In (\ref{['fig:survey:approx_vol']}), we show a volume-based approximation. The surface is represented as an appropriate level set of an occupancy function.
• Figure 5: Synthetic and real point clouds: Real world point cloud acquisitions (\ref{['fig:ign:mvs']},\ref{['fig:ign:lidar']}) have defects such as missing data from occlusion. However, surface reconstruction methods are often tested on point clouds that are produced by directly and uniformly sampling a ground truth surface (\ref{['fig:ign:uniform']}). While this sampling strategy allows to add artificial noise, it cannot realistically model missing data from occlusion. Instead, we test methods on synthetic MVS (\ref{['fig:ign:synmvs']}) and synthetic range scans (\ref{['fig:ign:synlidar']}) which allows us to reproduce realistic sampling defects.