Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TailorMe: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model

Stephan Wenninger, Fabian Kemper, Ulrich Schwanecke, Mario Botsch

TL;DR

This work registers a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database, and uses this data to learn an anatomically constrained volumetric human shape model in a self‐supervised fashion.

Abstract

Human shape spaces have been extensively studied, as they are a core element of human shape and pose inference tasks. Classic methods for creating a human shape model register a surface template mesh to a database of 3D scans and use dimensionality reduction techniques, such as Principal Component Analysis, to learn a compact representation. While these shape models enable global shape modifications by correlating anthropometric measurements with the learned subspace, they only provide limited localized shape control. We instead register a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database. We further enlarge our training data to the full Cartesian product of all skeletons and all soft tissues using physically plausible volumetric deformation transfer. This data is then used to learn an anatomically constrained volumetric human shape model in a self-supervised fashion. The resulting TailorMe model enables shape sampling, localized shape manipulation, and fast inference from given surface scans.

TailorMe: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model

TL;DR

This work registers a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database, and uses this data to learn an anatomically constrained volumetric human shape model in a self‐supervised fashion.

Abstract

Human shape spaces have been extensively studied, as they are a core element of human shape and pose inference tasks. Classic methods for creating a human shape model register a surface template mesh to a database of 3D scans and use dimensionality reduction techniques, such as Principal Component Analysis, to learn a compact representation. While these shape models enable global shape modifications by correlating anthropometric measurements with the learned subspace, they only provide limited localized shape control. We instead register a volumetric anatomical template, consisting of skeleton bones and soft tissue, to the surface scans of the CAESAR database. We further enlarge our training data to the full Cartesian product of all skeletons and all soft tissues using physically plausible volumetric deformation transfer. This data is then used to learn an anatomically constrained volumetric human shape model in a self-supervised fashion. The resulting TailorMe model enables shape sampling, localized shape manipulation, and fast inference from given surface scans.
Paper Structure (23 sections, 12 equations, 10 figures)

This paper contains 23 sections, 12 equations, 10 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Human Shape Models
  4. Modifying Virtual Humans
  5. Anatomical Models
  6. Training Data
  7. Skin and Skeleton Registration
  8. Volumetric Deformation Transfer
  9. Model Learning
  10. Network architecture
  11. Cross-correlation Loss
  12. Post-Processing
  13. Face Symmetrizing & Smoothing
  14. Intersection Avoidance
  15. Embedding High-resolution Skeleton
  16. ...and 8 more sections

Figures (10)

  • Figure 1: TailorMe: Our anatomically constrained volumetric human shape model allows to infer skeleton shape from a given surface scan. Due to injecting anthropometric measurements into the latent code of our model, we can then locally manipulate both the skeleton shape and the soft tissue distribution of a given person.
  • Figure 2: Male and female template model. In this work, we derive an additional skeleton layer that wraps the high resolution skeleton mesh and shares the triangulation with the skin layer.
  • Figure 3: Transferring the soft tissue of the left model onto the skeleton of the center-left model using surface-based deformation transfer, the skeleton wrap protrudes the skin (center-right). Our volumetric deformation transfer successfully avoids these artifacts (right).
  • Figure 4: Exemplary results of transferring the soft tissue of various people (top row) onto a single target skeleton via volumetric deformation transfer (bottom row). Note that soft tissue characteristics of the top row and skeletal dimensions of the bottom row are faithfully preserved.
  • Figure 5: Representing skeletons and skins in PCA subspaces separates their parameters, but the global nature of PCA prevents localized changes: Increasing the arm length of the left model also causes the body height to increase (right).
  • ...and 5 more figures