EGP3D: Edge-guided Geometric Preserving 3D Point Cloud Super-resolution for RGB-D camera

TL;DR

RGB-D point clouds from common sensors suffer from low resolution and blurry boundaries. EGP3D introduces an edge-guided, geometry-preserving upsampling pipeline that densifies point clouds and enforces edge alignment by projecting to a 2D space, computing a concave hull, and optimizing against RGB edges with three losses: $L_{ ext{CD}}$, $L_{ ext{HD}}$, and $L_{ ext{GS}}$, combined as $L = \alpha L_{ ext{CD}} + \beta L_{ ext{HD}} + \gamma L_{ ext{GS}}$. The method integrates a PU network based on Local Distance Indicator with an edge-guided module that operates in a unified coordinate frame, enabling direct 3D refinement without depth-to-point conversion. A real-world RGB-D dataset capturing noise and stray-light effects supports evaluation, and experiments show superior edge preservation and boundary fidelity compared to state-of-the-art PU and GDSR approaches, across multiple scales. Overall, EGP3D advances practical high-fidelity 3D reconstruction for RGB-D applications by combining edge-aware geometry preservation with realistic data and robust optimization.”

Abstract

Point clouds or depth images captured by current RGB-D cameras often suffer from low resolution, rendering them insufficient for applications such as 3D reconstruction and robots. Existing point cloud super-resolution (PCSR) methods are either constrained by geometric artifacts or lack attention to edge details. To address these issues, we propose an edge-guided geometric-preserving 3D point cloud super-resolution (EGP3D) method tailored for RGB-D cameras. Our approach innovatively optimizes the point cloud with an edge constraint on a projected 2D space, thereby ensuring high-quality edge preservation in the 3D PCSR task. To tackle geometric optimization challenges in super-resolution point clouds, particularly preserving edge shapes and smoothness, we introduce a multi-faceted loss function that simultaneously optimizes the Chamfer distance, Hausdorff distance, and gradient smoothness. Existing datasets used for point cloud upsampling are predominantly synthetic and inadequately represent real-world scenarios, neglecting noise and stray light effects. To address the scarcity of realistic RGB-D data for PCSR tasks, we built a dataset that captures real-world noise and stray-light effects, offering a more accurate representation of authentic environments. Validated through simulations and real-world experiments, the proposed method exhibited superior performance in preserving edge clarity and geometric details.

Figures (8)

• Figure 1: Due to depth sensor resolution limitations, the captured point cloud has limited pixel counts (left). Our pipeline projects the point cloud into 2D space (center) and utilizes RGB image edge information for upsampling guidance. This enables us to reconstruct a high-fidelity, super-resolved point cloud (right) using a geometric preserving algorithm.
• Figure 2: The pipeline of our method starts with a PU model li2024learning to increase point density, followed by the PCSR model for edge geometric optimization. The core innovation, the Edge-Guided Module, enhances point cloud boundaries using RGB edge information and refines the super-resolution process. The figure’s lower part illustrates how cascaded MLP blocks in both models progressively refine the point cloud.
• Figure 3: Qualitative comparisons of 4$\times$ upsampling results on the EGP3D dataset. Our method generates point clouds with clearer and more complete boundaries than state-of-the-art methods.
• Figure 4: The rotated point cloud visualization. From different viewpoints, it can still be observed that the point cloud generated by our method outperforms others in terms of resolution and the continuity of boundaries.
• Figure 5: A part of the EGP3D dataset showcasing point clouds and their corresponding RGB images, including: Geometric Objects, Fruit Models, and Complex Objects.