SR-CurvANN: Advancing 3D Surface Reconstruction through Curvature-Aware Neural Networks

TL;DR

The proposed SR-CurvANN (Surface Reconstruction Based on Curvature-Aware Neural Networks) incorporates neural network-based 2D inpainting to effectively reconstruct 3D surfaces, making it possible to learn and generalize patterns from a wide variety of training 3D models, generating comprehensive inpainted curvature images and surfaces.

Abstract

Incomplete or missing data in three-dimensional (3D) models can lead to erroneous or flawed renderings, limiting their usefulness in applications such as visualization, geometric computation, and 3D printing. Conventional surface-repair techniques often fail to infer complex geometric details in missing areas. Neural networks successfully address hole-filling tasks in 2D images using inpainting techniques. The combination of surface reconstruction algorithms, guided by the model's curvature properties and the creativity of neural networks in the inpainting processes should provide realistic results in the hole completion task. In this paper, we propose a novel method entitled SR-CurvANN (Surface Reconstruction Based on Curvature-Aware Neural Networks) that incorporates neural network-based 2D inpainting to effectively reconstruct 3D surfaces. We train the neural networks with images that represent planar representations of the curvature at vertices of hundreds of 3D models. Once the missing areas have been inferred, a coarse-to-fine surface deformation process ensures that the surface fits the reconstructed curvature image. Our proposal makes it possible to learn and generalize patterns from a wide variety of training 3D models, generating comprehensive inpainted curvature images and surfaces. Experiments conducted on 959 models with several holes have demonstrated that SR-CurvANN excels in the shape completion process, filling holes with a remarkable level of realism and precision.

Figures (14)

• Figure 1: Comparison of different shape completion techniques. The ground truth mesh with holes (leftmost) is shown alongside various reconstruction methods. Smooth reconstruction using a popular approach liepa2003filling, Poisson method kazhdan2013screened and Ramesh method centin2015rameshcleaner (these methods offer the best mesh distance measures for the hole filling task) and SR-CurvANN (rightmost). The proposed method demonstrates superior fidelity in filling complex holes, particularly in regions with intricate details, as seen in the comparison with other techniques.
• Figure 2: Overview of SR-CurvANN.1) Creation of curvatures images and holes dataset. We use a huge collection of 3D models to make a dataset of more than a million images of curvature, adding random hole masks to these images. 2) Training curvature-aware inpainting NNs. We use this database of images to teach inpainting artificial neural networks how to fix holes in a specific color palette spectrum. 3) Inpainting over masked curvature image. Once they are taught, these ad-hoc neural networks can infer missing parts of curvature images, prompting requests for them to inpaint images extracted from 3D meshes previously unused. 4) Deformation Algorithm. An automatic curvature-guided surface reconstruction algoritm is applied on the newly made surface patch until it matches the desired outcome perfectly.
• Figure 3: The distribution of the number of polygons in the dataset of 3D models.
• Figure 4: The initial 3D mesh model follows a farthest point sampling strategy, selecting the starting vertices for the perforations to ensure that each vertex is maximally distant from other vertices in the surface.
• Figure 5: Examples of some ground truth models in the dataset and the simulated holes.