Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery

Hongsheng Wang, Weiyue Zhang, Sihao Liu, Xinrui Zhou, Jing Li, Zhanyun Tang, Shengyu Zhang, Fei Wu, Feng Lin

TL;DR

The Hierarchical Graph Human Gaussian Control (HUGS) framework is introduced, which involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology, thereby enabling the capture of the complex geometrical and topological associations among body parts.

Abstract

Although 3D Gaussian Splatting (3DGS) has recently made progress in 3D human reconstruction, it primarily relies on 2D pixel-level supervision, overlooking the geometric complexity and topological relationships of different body parts. To address this gap, we introduce the Hierarchical Graph Human Gaussian Control (HUGS) framework for achieving high-fidelity 3D human reconstruction. Our approach involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology, thereby enabling the capture of the complex geometrical and topological associations among body parts. Additionally, we disentangle high-frequency features from global human features to refine surface details in body parts. Extensive experiments demonstrate that our method exhibits superior performance in human body reconstruction, particularly in enhancing surface details and accurately reconstructing body part junctions. Codes are available at https://wanghongsheng01.github.io/HUGS/.

Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery

TL;DR

The Hierarchical Graph Human Gaussian Control (HUGS) framework is introduced, which involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology, thereby enabling the capture of the complex geometrical and topological associations among body parts.

Abstract

Although 3D Gaussian Splatting (3DGS) has recently made progress in 3D human reconstruction, it primarily relies on 2D pixel-level supervision, overlooking the geometric complexity and topological relationships of different body parts. To address this gap, we introduce the Hierarchical Graph Human Gaussian Control (HUGS) framework for achieving high-fidelity 3D human reconstruction. Our approach involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology, thereby enabling the capture of the complex geometrical and topological associations among body parts. Additionally, we disentangle high-frequency features from global human features to refine surface details in body parts. Extensive experiments demonstrate that our method exhibits superior performance in human body reconstruction, particularly in enhancing surface details and accurately reconstructing body part junctions. Codes are available at https://wanghongsheng01.github.io/HUGS/.
Paper Structure (16 sections, 7 equations, 5 figures, 4 tables)

This paper contains 16 sections, 7 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Statistical Parametric Human Body Model
  4. 3D Gaussian Human
  5. Methods
  6. Preliminary
  7. Inter-Semantic Kinematic Topology
  8. Intro-Surface Disentanglement
  9. Semantic-Awareness Optimization
  10. Experiments
  11. Datasets and Metrics
  12. Quantitative Results
  13. Qualitative Results
  14. Ablation Study
  15. T-Test
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Human Gaussian Control with Hierarchical Semantic Graphs. (a) is a human Gaussian point cloud with semantic labels. (b) is the rendering output of (a). (c) is the result of our method compared with other methods on the Monocap dataset. LPIPS* = LPIPS $\times$ 1000.
  • Figure 2: HUGS framework. We introduce Human Gaussian Control with Hierarchical Semantic Graphs (HUGS) as a method for generating Gaussian humans, ensuring both realistic human appearance and anatomical structure. The input initialized point cloud is mapped using the SKT module to establish a graph structure that encodes the semantic topological relationships between different body parts. High-frequency regions are identified by SD for each body part, and the density of Gaussian points is increased in these areas. Finally, the adjusted Gaussian points are rendered to produce the image output.
  • Figure 3: Qualitative results of ablation study on SAG module and IJG module on the 377 sequence of ZJU-Mocap dataset.
  • Figure 4: Results from our method and baseline methods on ZJU-MoCap and MonoCap. Our method has superior rendering quality.
  • Figure 5: We use the Canny algorithm and Fourier transform to extract the high-frequency regions of the experimental results. The details of the Canny algorithm and Fourier transform can be found in the supplementary section.