Unified Geometry and Color Compression Framework for Point Clouds via Generative Diffusion Priors

TL;DR

This work presents a novel, test-time unified framework for compressing both geometry and color in 3D point clouds by leveraging a pretrained diffusion prior (Point-E) and prompt tuning to produce sparse seeds. The pipeline uses patch division, seed optimization, and a loss that accounts for permutation invariance to enable accurate reconstruction through diffusion denoising, all without dataset-specific training. Across colored objects, indoor scenes, and geometry-only baselines, the method yields notable geometry gains and competitive color performance, with strong generalization and applicability to offline tasks. While compression time is higher due to prompt tuning, the approach eliminates the need for training on target datasets and benefits from potential speedups via optimization techniques.

Abstract

With the growth of 3D applications and the rapid increase in sensor-collected 3D point cloud data, there is a rising demand for efficient compression algorithms. Most existing learning-based compression methods handle geometry and color attributes separately, treating them as distinct tasks, making these methods challenging to apply directly to point clouds with colors. Besides, the limited capacities of training datasets also limit their generalizability across points with different distributions. In this work, we introduce a test-time unified geometry and color compression framework of 3D point clouds. Instead of training a compression model based on specific datasets, we adapt a pre-trained generative diffusion model to compress original colored point clouds into sparse sets, termed 'seeds', using prompt tuning. Decompression is then achieved through multiple denoising steps with separate sampling processes. Experiments on objects and indoor scenes demonstrate that our method has superior performances compared to existing baselines for the compression of geometry and color.

Figures (5)

• Figure 1: Differences between our framework and existing point cloud compression methods. Conventional geometry compression techniques (a) focus solely on compressing coordinates, leaving color information unhandled. Attribute compression methods (b) encode colors into binary codes but retain uncompressed coordinates to assist with color decompression. In contrast, our framework (c) simultaneously optimizes the compression of both coordinates and colors into sparse sets, or 'seeds.' These seeds are encoded into binary codes using a non-learning-based method and, after decoding, are used in a denoising process with a diffusion model for decompression.
• Figure 2: The complete pipeline of our method. During compression, the input point cloud $P$ is first normalized using the calculated center $c$ and radius $r$, then divided into $n$ patches $P_1, \dots, P_n$ according to the chosen compression level $L_l$ using Patch Division. Seeds $P_c^1, \dots, P_c^n$ for each patch are obtained through Weights Aggregation and optimized via prompt tuning using our proposed loss $L_{CDM}$ computed on the pre-trained up-sampling diffusion model. Subsequently, $L_l$, patch statistics (including patch resolutions), scales (center $c$ and radius $r$), and seeds $P_c = [P_c^1, \dots, P_c^n]$ are encoded into binary codes, which are then decoded for decompression. During decompression, the decoded seeds $\bar{P}_c$ are split into patches $\bar{P}_c^1, \dots, \bar{P}_c^n$ using patch division. Each patch $\bar{P}_c^i$ is up-sampled to its recorded resolution $num_i$ through the diffusion model’s denoising process, yielding $P_o^i$. The concatenated output $P_o = \{P_o^i | i = 1 \dots n\}$ is then denormalized to produce the final output $\hat{P}_o$ using the recorded scales $c$ and $r$.
• Figure 3: Qualitative comparisons on 3D objects with colors.
• Figure 4: Qualitative comparisons on 3D indoor scenes with colors.
• Figure 5: Qualitative comparisons on Noisy Real Scans.