OffsetOPT: Explicit Surface Reconstruction without Normals

TL;DR

OffsetOPT tackles explicit surface reconstruction from 3D point clouds without normals by combining a two-stage strategy: (i) train a transformer-based triangle-prediction network on uniformly distributed data to predict adjacent triangles from $K$-NN neighborhoods, and (ii) generalize to arbitrary point clouds by optimizing per-point offsets $\Delta\mathbf p_n$ while freezing the network, effectively aligning input geometry with the network's preferred distribution. The method achieves high surface quality and sharp-feature preservation on both small shapes and large-scale scenes, outperforming key baselines such as SPSR, NKSR, and several neural/computational methods, without relying on normals. The approach is robust to open surfaces and scales to dense scans, making explicit surface reconstruction from real-world point clouds more practical. Overall, OffsetOPT advances explicit reconstruction by delivering accurate, edge-preserving meshes with good generalization and scalability.

Abstract

Neural surface reconstruction has been dominated by implicit representations with marching cubes for explicit surface extraction. However, those methods typically require high-quality normals for accurate reconstruction. We propose OffsetOPT, a method that reconstructs explicit surfaces directly from 3D point clouds and eliminates the need for point normals. The approach comprises two stages: first, we train a neural network to predict surface triangles based on local point geometry, given uniformly distributed training point clouds. Next, we apply the frozen network to reconstruct surfaces from unseen point clouds by optimizing a per-point offset to maximize the accuracy of triangle predictions. Compared to state-of-the-art methods, OffsetOPT not only excels at reconstructing overall surfaces but also significantly preserves sharp surface features. We demonstrate its accuracy on popular benchmarks, including small-scale shapes and large-scale open surfaces.

Figures (9)

• Figure 1: We present OffsetOPT (Offset OPTimization) for explicit surface reconstruction from 3D point clouds, without the need for point normals. The prediction model is trained on synthetic meshes with supervision and then generalized to unseen point clouds through unsupervised optimization of per-point offsets. All surfaces in this figure are reconstructed using the same trained model with offset optimization. Our method achieves state-of-the-art performance in overall surface quality, sharp detail preservation, and scalability.
• Figure 2: Overview of the proposed OffsetOPT method. (a) provides examples of training samples from the ABC dataset, showing meshes with uniformly distributed points and equilateral triangles (zoom-in for a better view). (b) is the training of our triangle prediction network in a supervised manner, where ground-truth labels are established from adjacent triangles of each point in the training meshes. The network predicts surface triangles based on KNN neighborhoods of points. (c) is the offset optimization for surface reconstruction. For a point cloud $\{{\mathbf p}_n\}$, we optimize its offsets $\{\Delta{\mathbf p}_n\}$ by backpropagating the unsupervised prediction loss through the frozen network. For each offset update during optimization, the KNN geometry used by the network is recomputed with points $\{{\mathbf p}_n + \Delta{\mathbf p}_n\}$.
• Figure 3: Evolution of manifold edges during OffsetOPT reconstruction. In the top row, we illustrate the reconstruction process for a shape from Thingi10k, with manifold edges in black and non-manifold edges in red. Given the input point cloud in (a), the initial reconstruction contains a high percentage of non-manifold edges, as shown in the histogram in (b), where 'Edge Adjacency' refers to the number of faces adjacent to each edge. We note that manifold edges have an adjacency of no more than 2. After 20 iterations of offset optimization, the percentage of manifold edges increases significantly from 75% to 99% in (c), while the number of non-manifold edges diminishes. The final reconstructed mesh, with detailed triangulation around the belly, is shown in (d). The bottom row shows a similar effect in scene reconstruction from ScanNet. Comparing (f) and (g), iterative offset optimization substantially increases the number of manifold edges. In (h), we display the final reconstruction with detailed triangulation around a sink.
• Figure 4: Comparison of different reconstruction methods on a large-scale building from Matterport3D Matterport3D. (a) The ground-truth surface. (b) SPSR kazhdan2013screened reconstruction using ground-truth normals, showing strong oversmoothing. (c) NKSR huang2023neural reconstruction without normals, resulting in many disconnected components. (d) The improved NKSR reconstruction with ground-truth normals. (e) Our reconstruction with OffsetOPT, which captures fine details without requiring normals. Zoomed-in triangulation details are shown for each surface.
• Figure 5: The percentage of manifold edges in the reconstructed surfaces using zero and proposed initializations, with red for zero and blue for proposed. Results are shown for the FAUST and Thingi10k datasets. The $x$-axis in our plots indicates the number of samples, while the $y$-axis represents the percentage.