Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ShapeBoost: Boosting Human Shape Estimation with Part-Based Parameterization and Clothing-Preserving Augmentation

Siyuan Bian, Jiefeng Li, Jiasheng Tang, Cewu Lu

TL;DR

ShapeBoost is proposed, a new human shape recovery framework that achieves pixel-level alignment even for rare body shapes and high accuracy for people wearing different types of clothes and a clothing-preserving data augmentation module is proposed to generate realistic images with diverse body shapes and accurate annotations.

Abstract

Accurate human shape recovery from a monocular RGB image is a challenging task because humans come in different shapes and sizes and wear different clothes. In this paper, we propose ShapeBoost, a new human shape recovery framework that achieves pixel-level alignment even for rare body shapes and high accuracy for people wearing different types of clothes. Unlike previous approaches that rely on the use of PCA-based shape coefficients, we adopt a new human shape parameterization that decomposes the human shape into bone lengths and the mean width of each part slice. This part-based parameterization technique achieves a balance between flexibility and validity using a semi-analytical shape reconstruction algorithm. Based on this new parameterization, a clothing-preserving data augmentation module is proposed to generate realistic images with diverse body shapes and accurate annotations. Experimental results show that our method outperforms other state-of-the-art methods in diverse body shape situations as well as in varied clothing situations.

ShapeBoost: Boosting Human Shape Estimation with Part-Based Parameterization and Clothing-Preserving Augmentation

TL;DR

ShapeBoost is proposed, a new human shape recovery framework that achieves pixel-level alignment even for rare body shapes and high accuracy for people wearing different types of clothes and a clothing-preserving data augmentation module is proposed to generate realistic images with diverse body shapes and accurate annotations.

Abstract

Accurate human shape recovery from a monocular RGB image is a challenging task because humans come in different shapes and sizes and wear different clothes. In this paper, we propose ShapeBoost, a new human shape recovery framework that achieves pixel-level alignment even for rare body shapes and high accuracy for people wearing different types of clothes. Unlike previous approaches that rely on the use of PCA-based shape coefficients, we adopt a new human shape parameterization that decomposes the human shape into bone lengths and the mean width of each part slice. This part-based parameterization technique achieves a balance between flexibility and validity using a semi-analytical shape reconstruction algorithm. Based on this new parameterization, a clothing-preserving data augmentation module is proposed to generate realistic images with diverse body shapes and accurate annotations. Experimental results show that our method outperforms other state-of-the-art methods in diverse body shape situations as well as in varied clothing situations.
Paper Structure (44 sections, 22 equations, 13 figures, 7 tables)

This paper contains 44 sections, 22 equations, 13 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. 3D Human Pose and Shape (HPS)
  4. Estimating 3D Body Shape
  5. Training datasets for human shape estimation.
  6. Estimation Framework.
  7. Method
  8. Preliminary
  9. SMPL Model.
  10. Part-based Parameterization
  11. ShapeBoost
  12. Clothing-preserving Image Transformation.
  13. Shape-parameter Derivation.
  14. Training Pipeline and Loss Function
  15. Shape Loss.
  16. ...and 29 more sections

Figures (13)

  • Figure 2: Previous SOTA methods for human shape estimation sengupta2021hierarchicalchoutas2022accurate (b, c) either fail on images of people wearing thick clothes or fail on images of people with extreme body shapes, while our method (d) achieves pixel-aligned results with high accuracy in both situations. Warmer colors on the human mesh represent higher per-vertex error.
  • Figure 3: The overall pipeline. First, the input image is randomly transformed with the clothing-preserving image transformation, and a convolutional neural network (CNN) is employed to extract skeleton, part widths and twist rotations. Then, the pose is obtained using inverse kinematics and the shape is obtained with our semi-analytical algorithm. The final mesh is retrieved based on the pose and shape parameter. The ShapeBoost framework consists of the image augmentation module and the shape reconstruction module.
  • Figure 4: Illustration of the shape decomposition procedure. From left to right, the figure shows the part segmentation, the definition of bone length and vertex width, and the slicing of one body part.
  • Figure 5: The illustration of the clothing-preserving transformation.
  • Figure 6: Qualitative results on SSP-3D and HBW datasets. From left to right: Input image, (a) Sengupta et al. sengupta2021hierarchical results, (b) SHAPY choutas2022accurate results, and (c) Our results. Warmer colors mean higher per-vertex error. Experiments on SSP-3D dataset use PVE-T-SC metric, and experiments on HBW dataset use P2P$_{20\mathrm{K}}$ metric.
  • ...and 8 more figures