Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents

Yingjie Zhou, Zicheng Zhang, Farong Wen, Jun Jia, Yanwei Jiang, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

TL;DR

This paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods, and reveals the presence of 6 common distortion categories in the generated 3DGCs.

Abstract

Although 3D generated content (3DGC) offers advantages in reducing production costs and accelerating design timelines, its quality often falls short when compared to 3D professionally generated content. Common quality issues frequently affect 3DGC, highlighting the importance of timely and effective quality assessment. Such evaluations not only ensure a higher standard of 3DGCs for end-users but also provide critical insights for advancing generative technologies. To address existing gaps in this domain, this paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods. During the dataset's construction, 50 fixed prompts are utilized to generate contents across all methods, resulting in the creation of 313 textured meshes that constitute the 3DGCQA dataset. The visualization intuitively reveals the presence of 6 common distortion categories in the generated 3DGCs. To further explore the quality of the 3DGCs, subjective quality assessment is conducted by evaluators, whose ratings reveal significant variation in quality across different generation methods. Additionally, several objective quality assessment algorithms are tested on the 3DGCQA dataset. The results expose limitations in the performance of existing algorithms and underscore the need for developing more specialized quality assessment methods. To provide a valuable resource for future research and development in 3D content generation and quality assessment, the dataset has been open-sourced in https://github.com/zyj-2000/3DGCQA.

3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents

TL;DR

This paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods, and reveals the presence of 6 common distortion categories in the generated 3DGCs.

Abstract

Although 3D generated content (3DGC) offers advantages in reducing production costs and accelerating design timelines, its quality often falls short when compared to 3D professionally generated content. Common quality issues frequently affect 3DGC, highlighting the importance of timely and effective quality assessment. Such evaluations not only ensure a higher standard of 3DGCs for end-users but also provide critical insights for advancing generative technologies. To address existing gaps in this domain, this paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods. During the dataset's construction, 50 fixed prompts are utilized to generate contents across all methods, resulting in the creation of 313 textured meshes that constitute the 3DGCQA dataset. The visualization intuitively reveals the presence of 6 common distortion categories in the generated 3DGCs. To further explore the quality of the 3DGCs, subjective quality assessment is conducted by evaluators, whose ratings reveal significant variation in quality across different generation methods. Additionally, several objective quality assessment algorithms are tested on the 3DGCQA dataset. The results expose limitations in the performance of existing algorithms and underscore the need for developing more specialized quality assessment methods. To provide a valuable resource for future research and development in 3D content generation and quality assessment, the dataset has been open-sourced in https://github.com/zyj-2000/3DGCQA.
Paper Structure (14 sections, 1 equation, 5 figures, 2 tables)

This paper contains 14 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D AIGC
  4. 3D Quality Assessment
  5. Database Construction
  6. Prompts
  7. Selected Generation Methods
  8. Visualization and Distortions
  9. Subjective Experiment
  10. Subjective Experiment Analysis
  11. Benchmark Experiment
  12. Experimental Setup
  13. Performance & Analysis
  14. Conclusion

Figures (5)

  • Figure 1: Differences in time cost and quality between 3D PGCs and AIGCs.
  • Figure 2: Details of the selected prompts and methods. (a) Word clouds consisting of prompts. (b) Number of 3DGCs successfully generated by each method. (c) Generation time of each method on RTX A6000 GPU.
  • Figure 3: Overview of 3DGCs in the 3DGCQA database and corresponding category labels.
  • Figure 4: Illustration of common distortions occurred in 3DGCs.
  • Figure 5: The evaluation interface used and distribution of subjective scores.