Perception-Guided Quality Metric of 3D Point Clouds Using Hybrid Strategy

TL;DR

A perception-guided hybrid metric (PHM) that adaptively leverages two visual strategies with respect to distortion degree to predict point cloud quality and achieves state-of-the-art (SOTA) performance and offers significant performance improvement in multiple distortion environments.

Abstract

Full-reference point cloud quality assessment (FR-PCQA) aims to infer the quality of distorted point clouds with available references. Most of the existing FR-PCQA metrics ignore the fact that the human visual system (HVS) dynamically tackles visual information according to different distortion levels (i.e., distortion detection for high-quality samples and appearance perception for low-quality samples) and measure point cloud quality using unified features. To bridge the gap, in this paper, we propose a perception-guided hybrid metric (PHM) that adaptively leverages two visual strategies with respect to distortion degree to predict point cloud quality: to measure visible difference in high-quality samples, PHM takes into account the masking effect and employs texture complexity as an effective compensatory factor for absolute difference; on the other hand, PHM leverages spectral graph theory to evaluate appearance degradation in low-quality samples. Variations in geometric signals on graphs and changes in the spectral graph wavelet coefficients are utilized to characterize geometry and texture appearance degradation, respectively. Finally, the results obtained from the two components are combined in a non-linear method to produce an overall quality score of the tested point cloud. The results of the experiment on five independent databases show that PHM achieves state-of-the-art (SOTA) performance and offers significant performance improvement in multiple distortion environments. The code is publicly available at https://github.com/zhangyujie-1998/PHM.

Figures (17)

• Figure 1: The core idea of this paper. For the high quality sample in the left box, the HVS tends to use point-wise comparison to detect microscopic differences. For the low quality sample in the right box, the HVS tends to use structure-wise comparison to measure appearance degradation at a large scale.
• Figure 2: Two examples of texture masking effect (provided by SJTU-PCQA yang2020predicting). (a) Two regions with different texture characteristics impaired by different intensities of texture noise. (b) Two samples impaired by the same intensity of texture noise.
• Figure 3: Investigation of the texture masking effect in FR-PCQA. (a) Scatter plot of MOS vs. $\rm PSNR_{Y}$. (b) Plot of fitted curves and each curve corresponds to one reference point cloud.
• Figure 4: Flowchart illustration of the proposed hybrid quality metric for point clouds, where $\mathbf{X}$ and $\mathbf{Y}$ denote the reference and distorted point clouds, respectively. To simulate the dynamic strategy adpoted by the HVS, the metric contains three parts: visible difference measurement, appearance degradation measurement, and adaptive combination.
• Figure 5: Exemplified examples of the proposed texture complexity evaluation: (a) "Longdress"; (b) "Soldier"; (c) "Sarah"; (d) "Ricardo". The residual error map and the texture complexity value are provided for each sample. Best viewed when zoomed in.