PointPCA: Point Cloud Objective Quality Assessment Using PCA-Based Descriptors

TL;DR

PointPCA tackles perceptual quality assessment for point clouds by introducing PCA-based descriptors for geometry and texture within a full-reference framework. It derives 23 geometric and 8 textural descriptors per point, computes 46 statistical features, compares reference and distorted clouds, and fuses 46 predictors with Random Forest regression to yield a single perceptual score. The evaluation across three subjective datasets shows PointPCA outperforming state-of-the-art metrics, with notable gains in mixed-distortion scenarios, and includes extensive analyses of parameter settings, color spaces, and regression models. Open-source software accompanies the work for practical deployment and further research.

Abstract

Point clouds denote a prominent solution for the representation of 3D photo-realistic content in immersive applications. Similarly to other imaging modalities, quality predictions for point cloud contents are vital for a wide range of applications, enabling trade-off optimizations between data quality and data size in every processing step from acquisition to rendering. In this work, we focus on use cases that consider human end-users consuming point cloud contents and, hence, we concentrate on visual quality metrics. In particular, we propose a set of perceptually relevant descriptors based on Principal Component Analysis (PCA) decomposition, which is applied to both geometry and texture data for full-reference point cloud quality assessment. Statistical features are derived from these descriptors to characterize local shape and appearance properties for both a reference and a distorted point cloud. The extracted statistical features are subsequently compared to provide corresponding predictions of visual quality for the distorted point cloud. As part of our method, a learning-based approach is proposed to fuse these individual predictors to a unified perceptual score. We validate the accuracy of the individual predictors, as well as the unified quality scores obtained after regression against subjectively annotated datasets, showing that our metric outperforms state-of-the-art solutions. Insights regarding design decisions are provided through exploratory studies, evaluating the performance of our metric under different parameter configurations, attribute domains, color spaces, and regression models. A software implementation of the proposed metric is made available at the following link: https://github.com/cwi-dis/pointpca.

Figures (6)

• Figure 1: PointPCA architecture: both the reference (i.e., Point cloud A) and the point cloud under evaluation (i.e., Point cloud B) are passing from the Duplicates Merging, computation of Descriptors, and computation of Statistical Features stages. After Duplicates Merging, the Correspondence between the two point clouds is computed and used for the Comparison of Statistical Features. A Predictor of visual quality is obtained per Statistical Feature, and all Predictors are finally fused to a total Quality Score through learning-based regression.
• Figure 2: The point cloud longdress (Figure \ref{['fig:featsMap0']}) and its statistical features using the mean and standard deviation of linearity (Figures \ref{['fig:featsMap1']}-\ref{['fig:featsMap2']}), planarity (Figures \ref{['fig:featsMap3']}-\ref{['fig:featsMap4']}), and first eigenvalue on texture (Figures \ref{['fig:featsMap7']}-\ref{['fig:featsMap8']}) descriptors. The amplitudes of statistical features are color-mapped, with red indicating higher and blue lower values. It can be noticed that the mean of linearity (\ref{['fig:featsMap1']}) and planarity (\ref{['fig:featsMap3']}) capture high- and low-frequency geometric regions, respectively. Moreover, the mean of the first eigenvalue on texture (\ref{['fig:featsMap7']}) highlights colorfulness. The standard deviation quantifies local dispersion, hence capturing high frequencies for all descriptors.
• Figure 3: Benchmarking of predictors by means of PLCC (thin opaque bars) and SROCC (thick transparent bars), grouped per descriptor $d_{u}^{\omega}$. Each bar represents a predictor, which relies on either the ${\mu}(d_{u}^{\omega})$ or the ${\sigma}(d_{u}^{\omega})$ statistical feature, and is indicated with blue and red color, respectively.
• Figure 4: Importance ranking scores of predictors, computed based on their average ranking order across all datasets, stacked per descriptor $d_{u}^{\omega}$. The ranking order is determined using both PLCC and SROCC. Blue and red bars represent predictors that rely on ${\mu}(d_{u}^{\omega})$ and ${\sigma}(d_{u}^{\omega})$, respectively.
• Figure 5: SROCC for every predictor $s_{j}$ and average SROCC for the total quality score $q$ in every dataset, under different neighborhood sizes using the $k$-nn algorithm with $k = \lbrace 9, 25, 49, 81 \rbrace$ to compute statistical features, and the $r$-search with $r = 0.008 \times B_{R}$ to compute descriptors.