Self-Supervised Large Scale Point Cloud Completion for Archaeological Site Restoration

TL;DR

This work tackles the challenge of completing large-scale, highly incomplete archaeological point clouds lacking ground-truth data. It introduces a novel MCOP representation that converts 3D data into a single, multi-channel 2D image, enabling large-scale point cloud completion via patch-based self-supervision and inpainting, followed by reprojection and Poisson reconstruction. The approach employs adversarial patch-based training, consistency and texture-sim losses to overcome unbalanced missing-data distributions and to preserve appearance and geometry. Experiments on Peruvian archaeological sites demonstrate state-of-the-art performance in both image-level and point-cloud metrics, producing high-fidelity, colored restorations of structures that are largely missing surfaces, and scalable to millions of points with robust visual texture.

Abstract

Point cloud completion helps restore partial incomplete point clouds suffering occlusions. Current self-supervised methods fail to give high fidelity completion for large objects with missing surfaces and unbalanced distribution of available points. In this paper, we present a novel method for restoring large-scale point clouds with limited and imbalanced ground-truth. Using rough boundary annotations for a region of interest, we project the original point clouds into a multiple-center-of-projection (MCOP) image, where fragments are projected to images of 5 channels (RGB, depth, and rotation). Completion of the original point cloud is reduced to inpainting the missing pixels in the MCOP images. Due to lack of complete structures and an unbalanced distribution of existing parts, we develop a self-supervised scheme which learns to infill the MCOP image with points resembling existing "complete" patches. Special losses are applied to further enhance the regularity and consistency of completed MCOP images, which is mapped back to 3D to form final restoration. Extensive experiments demonstrate the superiority of our method in completing 600+ incomplete and unbalanced archaeological structures in Peru.

Figures (9)

• Figure 3: Comparison of distribution of available points for incomplete point clouds suffering from occlusion/partial view and missing surfaces. The distributions are quite different and that of our outdoor sites is clearly vertically unbalanced.
• Figure 4: General pipeline of our MCOP based point cloud completion method. The input point cloud is first projected onto a 2D image with 5 channels (RGBD + rotation) using our MCOP representation. During training, a random sampling function $f_\theta$ is applied on the MCOP image to extract a group of local windows, which are trained against mostly complete local patches extracted from the dataset of all structures in an adversarial way. For inference, the MCOP image and an outline of the desired shape (e.g., wall height) is passed to our inpainting network adapted from LaMa. The final completed point cloud is obtained by remapping the inpainted MCOP image back into 3D.
• Figure 5: MCOP MCOP representation. For (a), we sample along a circular path while MCOP camera captures scan lines perpendicular to the path (b). Instead of the traditional MCOP image (c), we capture a 5 channel image (d, e, f).
• Figure 6: Histogram of random point cloud samples from the Mawchu Llacta dataset, with high completion ratios at the top to low-completion ratios at the bottom.
• Figure 7: Point cloud completion results of different methods on Mawchu Raw. We overlay completion onto input. More visualization results can be found in the supplemental.