ROI-Guided Point Cloud Geometry Compression Towards Human and Machine Vision

TL;DR

The paper tackles the challenge of compressing point cloud geometry without sacrificing downstream machine perception. It introduces ROI-guided point cloud geometry compression (RPCGC) with a base layer for coarse geometry and an enhancement layer for ROI-weighted residuals, supervised by ROI masks through RAM, RPN, and RSN. The approach incorporates a weighted rate-distortion objective, $L_{total}=L_r+L_v+L_t$, and a mask-aware distortion $D_{RW-CD}$ to prioritize informative regions, along with a detection loss to jointly optimize for machine perception. Experimental results on ScanNet and SUN RGB-D show improved high-bitrate detection accuracy (about a 10% gain over some learning-based baselines) and strong RD performance, though performance at low bitrates can be challenging; the method also demonstrates practical encoding/decoding times and memory efficiency. Overall, RPCGC advances joint human-machine vision compression by coupling ROI-driven semantic refinement with dual-layer coding and task-aware optimization, with implications for durable high-rate LiDAR data handling.

Abstract

Point cloud data is pivotal in applications like autonomous driving, virtual reality, and robotics. However, its substantial volume poses significant challenges in storage and transmission. In order to obtain a high compression ratio, crucial semantic details usually confront severe damage, leading to difficulties in guaranteeing the accuracy of downstream tasks. To tackle this problem, we are the first to introduce a novel Region of Interest (ROI)-guided Point Cloud Geometry Compression (RPCGC) method for human and machine vision. Our framework employs a dual-branch parallel structure, where the base layer encodes and decodes a simplified version of the point cloud, and the enhancement layer refines this by focusing on geometry details. Furthermore, the residual information of the enhancement layer undergoes refinement through an ROI prediction network. This network generates mask information, which is then incorporated into the residuals, serving as a strong supervision signal. Additionally, we intricately apply these mask details in the Rate-Distortion (RD) optimization process, with each point weighted in the distortion calculation. Our loss function includes RD loss and detection loss to better guide point cloud encoding for the machine. Experiment results demonstrate that RPCGC achieves exceptional compression performance and better detection accuracy (10% gain) than some learning-based compression methods at high bitrates in ScanNet and SUN RGB-D datasets.

Figures (5)

• Figure 1: The overview of our proposed RPCGC framework includes: (1) The Residual Analysis Module (RAM) shows the coordinate coarse coding process. (2) The ROI prediction network (RPN) is the probability prediction network. (3) The ROI Searching Network (RSN) involves processing the predicted mask and applying it as a weight to the residual features. FAM denotes Feature Alignment Module. Residual Synthesis Module (RSM) denotes the decoding process. When the original point cloud is reconstructed, we then feed the reconstructed data into the detection network for further analysis. $Q$ and $Q^{-1}$ mean quantization and de-quantization. $\ominus$ and $\oplus$ denote tensor element-wise subtraction and addition operation.
• Figure 2: The overview of the network in RPCGC. (a) "RAM" stands for the Residual Analysis Module, (b) "RSM" denotes the Residual Synthesis Module. (c) "ResBlockA" refers to the residual structure employed in RPN. (d) "ResBlockB" describes the residual structure in SAM cheng2020learned. (e) "SAM" represents the Semantic-aware Attention Module. (f) "IRN" signifies the Inception-Residual Network. (g) "MSFEM" stands for the Multi-Scale Feature Extraction Module. All the convolutions in the network are 3D sparse convolutions. $N$ represents the dimension of the input point cloud, and $C$ denotes the number of channels.
• Figure 3: Performance comparison using Rate-Distortion (RD) and Rate-Detection (R-mAP) curves under different bitrates. (a), (e), (b), and (f) show the RD curves on the ScanNet and SUN RGB-D, and the RPCGC-base represents a scenario where no optimization strategies are added. Meanwhile, (c), (g), (d), and (h) present the R-mAP curves on the ScanNet and SUN RGB-D.
• Figure 4: The visualization of the detection outputs of different compression algorithms on ScanNet dataset, where the bpp and PSNR represent the average values.
• Figure 5: The ablation experiment. (a) and (b) show the detection results of RPCGC on ScanNet using various models in Group-Free. (c) and (d) are different strategies in RPCGC: "DL" denotes Detection Loss, "MCD" means Masking Chamfer Distance Loss, "MFM" signifies masking Residual Map, "SAM" denotes the Semantic-Aware Module, and "MSFEM" means the Multi-Scale Feature Extraction Module. (e-g) show the RD curves of RPCGC and other methods in the MPEG test dataset.