GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment

TL;DR

GMS-3DQA tackles the efficiency-question in projection-based 3D model quality assessment by introducing multi-projection grid mini-patch sampling to fuse six views into a single quality map. A Swin-Tiny backbone then regresses this map to perceptual quality, achieving superior performance over state-of-the-art NR-3DQA methods on several PCQA datasets while substantially reducing inference time. The approach demonstrates strong cross-database and cross-domain generalization and shows practical benefits for real-time or large-scale 3D quality monitoring. Overall, the method offers a scalable, fast, and robust NR-3DQA solution applicable to both point clouds and meshes, with public code available.

Abstract

Nowadays, most 3D model quality assessment (3DQA) methods have been aimed at improving performance. However, little attention has been paid to the computational cost and inference time required for practical applications. Model-based 3DQA methods extract features directly from the 3D models, which are characterized by their high degree of complexity. As a result, many researchers are inclined towards utilizing projection-based 3DQA methods. Nevertheless, previous projection-based 3DQA methods directly extract features from multi-projections to ensure quality prediction accuracy, which calls for more resource consumption and inevitably leads to inefficiency. Thus in this paper, we address this challenge by proposing a no-reference (NR) projection-based \textit{\underline{G}rid \underline{M}ini-patch \underline{S}ampling \underline{3D} Model \underline{Q}uality \underline{A}ssessment (GMS-3DQA)} method. The projection images are rendered from six perpendicular viewpoints of the 3D model to cover sufficient quality information. To reduce redundancy and inference resources, we propose a multi-projection grid mini-patch sampling strategy (MP-GMS), which samples grid mini-patches from the multi-projections and forms the sampled grid mini-patches into one quality mini-patch map (QMM). The Swin-Transformer tiny backbone is then used to extract quality-aware features from the QMMs. The experimental results show that the proposed GMS-3DQA outperforms existing state-of-the-art NR-3DQA methods on the point cloud quality assessment databases. The efficiency analysis reveals that the proposed GMS-3DQA requires far less computational resources and inference time than other 3DQA competitors. The code will be available at https://github.com/zzc-1998/GMS-3DQA.

Figures (8)

• Figure 1: Illustration of the challenge for the projection-based 3DQA methods, where employing multi-projections can help improve the performance but inevitably consume more computational resources.
• Figure 2: The framework of the proposed method. Perpendicular projections are captured from the input point clouds corresponding to the six surfaces of the cube, which are further processed into mini-patch maps. Patches are randomly selected from the mini-patch maps and then spliced into quality mini-patch maps (QMMs) for evaluation. Finally, the quality-aware features are extracted by Swin-Transformer tiny and regressed into perceptual scores.
• Figure 3: Illustration of the projection process, where the projections are rendered from the cube-like six perpendicular viewpoints.
• Figure 4: An example of the multi-projection grid mini-patch sampling process. $\mathcal{L} \times \mathcal{L}$ ($7 \times 7$ for this example) uniform grids are generated from the multi-projections and mini-patches are sampled from the uniform grids. Then $\lfloor \mathcal{L}^2/6 \rfloor$ (8 for this example) mini-patches are randomly sampled from each projection's mini-patch map and the last mini-patch map provides 1 extra mini-patch to fill up the QMM. It's worth mentioning that the blank mini-patches are ignored.
• Figure 5: Examples of the distorted point clouds and their corresponding QMMs, where we can see the noise and the patterns are well-preserved and even more obvious in the QMMs. The VPCC and GPCC compression standards are issued by the MPEG group for point cloud compression schwarz2018emerging. More specifically, the VPCC compression introduces blur to the mini-patches while the GPCC compression causes more artifacts as well as block effect.