Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction

Kennard Yanting Chan, Fayao Liu, Guosheng Lin, Chuan Sheng Foo, Weisi Lin

TL;DR

Fine Structured-Aware Sampling (FSS), a new sampling training scheme to train pixel-aligned implicit models for single-view human reconstruction, and shows how normals of sample points can be capitalized in the training process to improve results.

Abstract

Pixel-aligned implicit models, such as PIFu, PIFuHD, and ICON, are used for single-view clothed human reconstruction. These models need to be trained using a sampling training scheme. Existing sampling training schemes either fail to capture thin surfaces (e.g. ears, fingers) or cause noisy artefacts in reconstructed meshes. To address these problems, we introduce Fine Structured-Aware Sampling (FSS), a new sampling training scheme to train pixel-aligned implicit models for single-view human reconstruction. FSS resolves the aforementioned problems by proactively adapting to the thickness and complexity of surfaces. In addition, unlike existing sampling training schemes, FSS shows how normals of sample points can be capitalized in the training process to improve results. Lastly, to further improve the training process, FSS proposes a mesh thickness loss signal for pixel-aligned implicit models. It becomes computationally feasible to introduce this loss once a slight reworking of the pixel-aligned implicit function framework is carried out. Our results show that our methods significantly outperform SOTA methods qualitatively and quantitatively. Our code is publicly available at https://github.com/kcyt/FSS.

Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction

TL;DR

Fine Structured-Aware Sampling (FSS), a new sampling training scheme to train pixel-aligned implicit models for single-view human reconstruction, and shows how normals of sample points can be capitalized in the training process to improve results.

Abstract

Pixel-aligned implicit models, such as PIFu, PIFuHD, and ICON, are used for single-view clothed human reconstruction. These models need to be trained using a sampling training scheme. Existing sampling training schemes either fail to capture thin surfaces (e.g. ears, fingers) or cause noisy artefacts in reconstructed meshes. To address these problems, we introduce Fine Structured-Aware Sampling (FSS), a new sampling training scheme to train pixel-aligned implicit models for single-view human reconstruction. FSS resolves the aforementioned problems by proactively adapting to the thickness and complexity of surfaces. In addition, unlike existing sampling training schemes, FSS shows how normals of sample points can be capitalized in the training process to improve results. Lastly, to further improve the training process, FSS proposes a mesh thickness loss signal for pixel-aligned implicit models. It becomes computationally feasible to introduce this loss once a slight reworking of the pixel-aligned implicit function framework is carried out. Our results show that our methods significantly outperform SOTA methods qualitatively and quantitatively. Our code is publicly available at https://github.com/kcyt/FSS.
Paper Structure (23 sections, 12 figures, 4 tables)

This paper contains 23 sections, 12 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Single-view Human Reconstruction
  4. PIFu and IntegratedPIFu
  5. Method
  6. Fine Structure-aware Sampling Scheme (FSS)
  7. 1. Twinned Sample Points
  8. 2. Proximity-adaptive Displacement
  9. 3. Anchor Sample Points
  10. 4. Counter Sample Points
  11. 5. Smplx-guided Sampling
  12. Exploiting Normals of Sample Points (NSP)
  13. Mesh Thickness Loss (MTL) Supervision
  14. Experiments
  15. Datasets
  16. ...and 8 more sections

Figures (12)

  • Figure 1: Unlike existing schemes, FSS can: 1. Adapts to thickness of mesh. 2. Prioritize regions that are challenging.
  • Figure 2: Unlike SOTA methods, our method captures thin body features (e.g. fingers, ears) w/o causing noisy, wavy artefacts.
  • Figure 3: Overview of our FSS sampling training scheme (with NSP and MTL) in a single architecture.
  • Figure 4: Four of the five key features in FSS. (a) Twinned Sample Points (b) Anchor Sample Points (c) Counter Sample Points (d) Smplx-guided Sampling.
  • Figure 5: Usefulness of sample points' normals
  • ...and 7 more figures