From Images to Point Clouds: An Efficient Solution for Cross-media Blind Quality Assessment without Annotated Training

TL;DR

This work tackles blind point-cloud quality assessment with no target-domain annotations by transferring rich IQA priors from images through a distortion-aware unsupervised domain adaptation framework. It decomposes the transfer objective into distortion-guided feature alignment and quality-sensitive maintenance, introducing distortion distribution weights and a contrastive disentanglement mechanism to preserve quality mapping. The proposed DWIT-PCQA achieves competitive performance against fully supervised PCQA methods and outperforms prior image-to-point-cloud transfer approaches, demonstrating the practical value of cross-media priors and robust domain alignment. This approach enables scalable PCQA without point-cloud labels and highlights a promising connection between IQA and PCQA across media modalities with potential for broader cross-domain quality evaluations.

Abstract

We present a novel quality assessment method which can predict the perceptual quality of point clouds from new scenes without available annotations by leveraging the rich prior knowledge in images, called the Distribution-Weighted Image-Transferred Point Cloud Quality Assessment (DWIT-PCQA). Recognizing the human visual system (HVS) as the decision-maker in quality assessment regardless of media types, we can emulate the evaluation criteria for human perception via neural networks and further transfer the capability of quality prediction from images to point clouds by leveraging the prior knowledge in the images. Specifically, domain adaptation (DA) can be leveraged to bridge the images and point clouds by aligning feature distributions of the two media in the same feature space. However, the different manifestations of distortions in images and point clouds make feature alignment a difficult task. To reduce the alignment difficulty and consider the different distortion distribution during alignment, we have derived formulas to decompose the optimization objective of the conventional DA into two suboptimization functions with distortion as a transition. Specifically, through network implementation, we propose the distortion-guided biased feature alignment which integrates existing/estimated distortion distribution into the adversarial DA framework, emphasizing common distortion patterns during feature alignment. Besides, we propose the quality-aware feature disentanglement to mitigate the destruction of the mapping from features to quality during alignment with biased distortions. Experimental results demonstrate that our proposed method exhibits reliable performance compared to general blind PCQA methods without needing point cloud annotations.

Figures (7)

• Figure 1: Comparison with existing ideas. The existing IT-PCQA directly aligns and maps the features into the unified distribution by domain adaptation (DA). The proposed DWIT-PCQA considers differential distortion distributions between the source domain and the target domain, and emphasizes common distortion patterns. Besides, contrastive learning is employed to promote the quality-sensitive representation during distortion-guided alignment. The dashed line represents the testing flow.
• Figure 2: Different distortion distributions of images and point clouds. Features of images and projections are extracted from the pre-trained backbone of HyperIQA Su2020Hyper. Red represents the feature distribution of images (TID2013 tid2013), while green represents the feature distribution of point cloud projections (SJTU-PCQA yang2020predicting). The numbers in the figure represent the distortion types numbering in their original papers.
• Figure 3: Performance change with each removal of distortion type in source domain of IT-PCQA (TID2013 as the source domain, and SJTU-PCQA as the target domain). The distortion type number is the same as in the original paper tid2013, shown in Table \ref{['tab:tiddistortions']}. Positive $\Delta PLCC$ represents positive performance improvement after removing this distortion, indicating that the removed distortion has a potential negative impact on DA.
• Figure 4: An illustration of our proposed DWIT-PCQA. It consists of the following substeps: (a) the preprocessing module casts the point clouds and images into the same data format; (b) the generative network extracts representative features from both source and target domains; (c) the conditional-discriminative network aligns the feature likelihood functions with respect to distortions $\mathcal{D} (\mathbf{z}|\mathbf{y}_d)$ between the source domain and the target domain; (d) the quality-aware feature disentanglement module is applied to harmonize the conflicts of feature likelihood functions with respect to distortions $\mathcal{D} (\mathbf{z}|\mathbf{y}_d)$ and perceptual quality $\mathcal{D} (\mathbf{z}|\mathbf{y})$; (e) the distortion classification network estimates the necessary distortion distribution for cross-domain feature alignment under biased distortions; (f) the quality regression network achieves quality prediction.
• Figure 5: Positive and negative sample configuration for the quality-aware feature disentanglement. Positive and negative samples are designed to simultaneously achieve two distributions related to quality and distortion in the same feature space. The patches of the same image are set to be positive samples of anchor images (with close perception), and the images with the same content but different distortion types are set to be negative samples (with different perception). In this way, the extracted features are promoted to be both quality-ware and distortion-aware.

Theorems & Definitions (2)

• Theorem 1
• proof