Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation

TL;DR

This work addresses the lack of large-scale, full-body digital human quality assessment data by introducing SJTU-H3D, a database with 40 high-quality references and 1,120 distorted samples across seven distortions. It proposes a zero-shot, no-reference DHQA approach that fuses semantic cues from CLIP, spatial quality via NIQE, and geometry cues from mesh dihedral angles into the DHQI score, where $Q_{DHQI} = Q_A + Q_N + Q_G$. The method demonstrates strong zero-shot performance, competes with supervised approaches on DHQA tasks, and reveals distortion-specific strengths and weaknesses, establishing a robust baseline for DHQA research. The dataset and code are publicly available, enabling broader evaluation and refinement of digital human quality assessment techniques.

Abstract

Digital humans have witnessed extensive applications in various domains, necessitating related quality assessment studies. However, there is a lack of comprehensive digital human quality assessment (DHQA) databases. To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans. It comprises 40 high-quality reference digital humans and 1,120 labeled distorted counterparts generated with seven types of distortions. The SJTU-H3D database can serve as a benchmark for DHQA research, allowing evaluation and refinement of processing algorithms. Further, we propose a zero-shot DHQA approach that focuses on no-reference (NR) scenarios to ensure generalization capabilities while mitigating database bias. Our method leverages semantic and distortion features extracted from projections, as well as geometry features derived from the mesh structure of digital humans. Specifically, we employ the Contrastive Language-Image Pre-training (CLIP) model to measure semantic affinity and incorporate the Naturalness Image Quality Evaluator (NIQE) model to capture low-level distortion information. Additionally, we utilize dihedral angles as geometry descriptors to extract mesh features. By aggregating these measures, we introduce the Digital Human Quality Index (DHQI), which demonstrates significant improvements in zero-shot performance. The DHQI can also serve as a robust baseline for DHQA tasks, facilitating advancements in the field. The database and the code are available at https://github.com/zzc-1998/SJTU-H3D.

Figures (9)

• Figure 1: Motovation of our works. Unlike 2D images/videos, collection of 3D digital humans is more difficult and expensive. Therefore there is a lack of subjective databases for 3D digital humans currently. To tackle this issue, we propose the first perceptual quality assessment database for full-body digital humans (SJTU-H3D). Furthermore, in contrast to the numerous large-scale image/video quality assessment (I/VQA) databases that facilitate data-driven methodologies, supervised methods can be easily bothered by the bias of the limited number of DHQA databases, affecting generalization ability. Thus we propose a zero-shot no-reference quality assessment method to address this concern.
• Figure 2: Overview of the reference 3D digital humans.
• Figure 3: Geometry information and colorfulness for the reference 3D digital humans.
• Figure 4: Examples of 3D digital humans with different types of distortions. Specifically, the presence of GN may introduce non-uniform irregular geometry artifacts to the model, while CN can give rise to noisy interference that negatively impacts visual quality. FS, on the other hand, can induce a simplification of the model's geometric representation, consequently leading to shape distortion. PC has the potential to generate blocky artifacts within the model. UMC can cause blurred texture misalignment, and TD can result in blurry and indistinct texture mapping. Additionally, TC may introduce color inaccuracies, while banding JPEG distortion can manifest as striped visual artifacts.
• Figure 5: Inllustration of the rendering process and rating interface.