A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion

TL;DR

The paper tackles the challenge of completing incomplete 3D point clouds by moving beyond Chamfer Distance-based training. It introduces a Conditional Point Diffusion-Refinement (PDR) framework that first generates a coarse, uniformly distributed completion via a conditional DDPM (CGNet) and then refines it with a dedicated Refinement Network (RFNet) in a dual-path architecture, enabling both global shape fidelity and sharp local details. By incorporating novel modules such as Point Adaptive Deconvolution (PA-Deconv) and a Feature Transfer pathway, the method achieves state-of-the-art results on standard benchmarks and can accelerate diffusion by up to 50× without substantial loss in quality. The approach also extends to controllable generation by conditioning on bounding boxes, highlighting its practical versatility for 3D scene understanding and reconstruction tasks.

Abstract

3D point cloud is an important 3D representation for capturing real world 3D objects. However, real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to non-uniform point cloud generation. To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet). The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation. DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation. The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud. Furthermore, we develop a novel dual-path architecture for both networks. The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces and sharp details. Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion. Remarkably, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.

Figures (12)

• Figure 1: Our Conditional Point Diffusion-Refinement (PDR) paradigm first moves a Gaussian noise step by step towards a coarse completion of the partial observation through a diffusion model (DDPM). Then it refines the coarse point cloud by one step to obtain a high quality point cloud.
• Figure 2: Network architecture of the Conditional Generation Network (CGNet) and ReFinement Network (RFNet). It consists of the Condition Feature Extraction subnet and the Denoise subnet.
• Figure 3: (a) Insert information of the diffusion step embedding and the global feature to the shared MLP. (b) The Feature Transfer module maps features from the incomplete point cloud to the noisy point cloud. (c) Refine and upsample the coarse points at the same time.
• Figure 4: Visual comparison of point cloud completion results on the MVP dataset ($16384$ points).
• Figure 5: Our method can be extended to controllable point cloud generation.