Representation Learning of Point Cloud Upsampling in Global and Local Inputs

TL;DR

To address sparse and noisy point clouds, this paper introduces ReLPU, a parallel global-local input learning framework for point cloud upsampling. ReLPU uses two parallel autoencoders to extract global and local features (via Average Segment and patch inputs), fusing them before a shared decoder, and it provides gradient-based saliency maps in spherical coordinates for interpretability. The approach is general and backbones-compatible, validated on PU1K and ABC-10k where it consistently improves geometric fidelity and robustness over state-of-the-art methods; ablations and visualizations confirm the benefit of parallel fusion and the interpretability of feature contributions. The work advances practical point cloud upsampling by delivering a flexible, explainable framework that can accommodate future backbone architectures.

Abstract

In recent years, point cloud upsampling has been widely applied in tasks such as 3D reconstruction and object recognition. This study proposed a novel framework, ReLPU, which enhances upsampling performance by explicitly learning from both global and local structural features of point clouds. Specifically, we extracted global features from uniformly segmented inputs (Average Segments) and local features from patch-based inputs of the same point cloud. These two types of features were processed through parallel autoencoders, fused, and then fed into a shared decoder for upsampling. This dual-input design improved feature completeness and cross-scale consistency, especially in sparse and noisy regions. Our framework was applied to several state-of-the-art autoencoder-based networks and validated on standard datasets. Experimental results demonstrated consistent improvements in geometric fidelity and robustness. In addition, saliency maps confirmed that parallel global-local learning significantly enhanced the interpretability and performance of point cloud upsampling.

Figures (9)

• Figure 1: Illustrates the structure of Traditional Upsampling Network (a) like MPU, PU-GCN, etc. And present schematic diagram of two data input methods as the transcendental local and global feature for ReLPU- (b) and ReLPU (c) models for point cloud upsampling. ReLPU- uses a sequential autoencoder for unified feature extraction, while ReLPU employs two parallel autoencoders to extract local and global features separately, enhancing upsampling performance.
• Figure 2: Examples of saliency maps return upsampling loss after multiply the radial distance from the point to the center in spherical coordinates.
• Figure 3: Visualizes saliency maps comparing ReLPU and original models (MPU, Dis-PU, PU-GCN, PUCRN) by patch (local) inputs and AS (global) inputs. Points are color-coded based on normalized saliency score ranks in spherical coordinates, where higher values represent higher saliency.
• Figure 4: Simple Linear regression of saliency maps comparing original models (MPU, Dis-PU, PU-GCN, PUCRN) by patch (local) inputs: Group 1; and AS (global) inputs: Group 2; and ReLPU: Group 3. Where $S_\theta(X) = - x_i * r$, the slope is represented as $x_i$. When $x_i$ tends towards 0, under the same $s_i$ with farther radial distance $r$ will have larger saliency maps score in $x_i$.
• Figure 5: Visualize the result of state-of-the-art methods (MPU, PU-GCN, Dis PU and PUCRN) with ReLPU in ($4\times$) upsampling on PU1K test data by using 2048 input points.