Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Report on Methods and Applications for Crafting 3D Humans

Lei Liu, Ke Zhao

TL;DR

The paper addresses the challenge of scalable, high fidelity 3D human model and avatar creation driven by AI, using diffusion, NeRF, and 3D scene representations to enable realistic rendering and animation. It surveys core 3D representations, neural rendering approaches, and diffusion-based generation, and introduces three text-to-3D avatar frameworks—DreamAvatar, DreamFace, and AvatarCraft—each advancing geometry, texture, and animatability. Key contributions include explicit representation discussions, diffusion training and sampling formulations, and novel architectures that integrate SMPL guidance with neural rendering and loss design. The work demonstrates broad practical impact across entertainment, VR/AR, education, healthcare, and consumer applications, while underscoring the potential and risks of large-scale models in 3D content generation and the need for ongoing innovation and responsible deployment.

Abstract

This paper presents an in-depth exploration of 3D human model and avatar generation technology, propelled by the rapid advancements in large-scale models and artificial intelligence. The paper reviews the comprehensive process of 3D human model generation, from scanning to rendering, and highlights the pivotal role these models play in entertainment, VR, AR, healthcare, and education. We underscore the significance of diffusion models in generating high-fidelity images and videos. It emphasizes the indispensable nature of 3D human models in enhancing user experiences and functionalities across various fields. Furthermore, this paper anticipates the potential of integrating large-scale models with deep learning to revolutionize 3D content generation, offering insights into the future prospects of the technology. It concludes by emphasizing the importance of continuous innovation in the field, suggesting that ongoing advancements will significantly expand the capabilities and applications of 3D human models and avatars.

Report on Methods and Applications for Crafting 3D Humans

TL;DR

The paper addresses the challenge of scalable, high fidelity 3D human model and avatar creation driven by AI, using diffusion, NeRF, and 3D scene representations to enable realistic rendering and animation. It surveys core 3D representations, neural rendering approaches, and diffusion-based generation, and introduces three text-to-3D avatar frameworks—DreamAvatar, DreamFace, and AvatarCraft—each advancing geometry, texture, and animatability. Key contributions include explicit representation discussions, diffusion training and sampling formulations, and novel architectures that integrate SMPL guidance with neural rendering and loss design. The work demonstrates broad practical impact across entertainment, VR/AR, education, healthcare, and consumer applications, while underscoring the potential and risks of large-scale models in 3D content generation and the need for ongoing innovation and responsible deployment.

Abstract

This paper presents an in-depth exploration of 3D human model and avatar generation technology, propelled by the rapid advancements in large-scale models and artificial intelligence. The paper reviews the comprehensive process of 3D human model generation, from scanning to rendering, and highlights the pivotal role these models play in entertainment, VR, AR, healthcare, and education. We underscore the significance of diffusion models in generating high-fidelity images and videos. It emphasizes the indispensable nature of 3D human models in enhancing user experiences and functionalities across various fields. Furthermore, this paper anticipates the potential of integrating large-scale models with deep learning to revolutionize 3D content generation, offering insights into the future prospects of the technology. It concludes by emphasizing the importance of continuous innovation in the field, suggesting that ongoing advancements will significantly expand the capabilities and applications of 3D human models and avatars.
Paper Structure (29 sections, 5 equations, 8 figures)

This paper contains 29 sections, 5 equations, 8 figures.

Table of Contents

  1. Introduction
  2. 3D Technologies
  3. 3D Representation
  4. Explicit Representations
  5. Meshes, Point Clouds and Voxels
  6. NeRF and 3DGS
  7. Diffusion Models
  8. Forward Process (Diffusion)
  9. Reverse Process (Denoising)
  10. Training Objective
  11. Sampling
  12. Avatar
  13. Text-to-3D avatar generation
  14. DreamAvatar
  15. Utilization of the SMPL Model:
  16. ...and 14 more sections

Figures (8)

  • Figure 1: DreamAvatar cao2023dreamavatar can generate high-quality geometry and texture for any type of human avatar.
  • Figure 2: DreamAvatar can handle and control the 3D generation with any pose.
  • Figure 3: Overview of DreamAvatar.
  • Figure 4: We show the pipeline of Dreamface zhang2023dreamface.
  • Figure 5: Geometry generation pipeline of DreamFace.
  • ...and 3 more figures