Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FuRPE: Learning Full-body Reconstruction from Part Experts

Zhaoxin Fan, Yuqing Pan, Hao Xu, Zhenbo Song, Zhicheng Wang, Kejian Wu, Hongyan Liu, Jun He

TL;DR

FuRPE tackles annotation scarcity in full-body reconstruction by leveraging part-experts to generate pseudo labels for body, head, and hands, paired with a simple pseudo ground-truth selection scheme. The method introduces an EMA self-supervision framework and an expert-derived feature distillation strategy to mitigate bias from pseudo labels while training. The approach trains both two-stage and single-stage SMPL-X networks and achieves substantial performance gains over baselines and state-of-the-art methods across multiple benchmarks. The work demonstrates that learning from expert-derived pseudo supervision can reshape full-body reconstruction, enabling robust, data-efficient learning on large-scale datasets.

Abstract

In the field of full-body reconstruction, the scarcity of annotated data often impedes the efficacy of prevailing methods. To address this issue, we introduce FuRPE, a novel framework that employs part-experts and an ingenious pseudo ground-truth selection scheme to derive high-quality pseudo labels. These labels, central to our approach, equip our network with the capability to efficiently learn from the available data. Integral to FuRPE is a unique exponential moving average training strategy and expert-derived feature distillation strategy. These novel elements of FuRPE not only serve to further refine the model but also to reduce potential biases that may arise from inaccuracies in pseudo labels, thereby optimizing the network's training process and enhancing the robustness of the model. We apply FuRPE to train both two-stage and fully convolutional single-stage full-body reconstruction networks. Our exhaustive experiments on numerous benchmark datasets illustrate a substantial performance boost over existing methods, underscoring FuRPE's potential to reshape the state-of-the-art in full-body reconstruction.

FuRPE: Learning Full-body Reconstruction from Part Experts

TL;DR

FuRPE tackles annotation scarcity in full-body reconstruction by leveraging part-experts to generate pseudo labels for body, head, and hands, paired with a simple pseudo ground-truth selection scheme. The method introduces an EMA self-supervision framework and an expert-derived feature distillation strategy to mitigate bias from pseudo labels while training. The approach trains both two-stage and single-stage SMPL-X networks and achieves substantial performance gains over baselines and state-of-the-art methods across multiple benchmarks. The work demonstrates that learning from expert-derived pseudo supervision can reshape full-body reconstruction, enabling robust, data-efficient learning on large-scale datasets.

Abstract

In the field of full-body reconstruction, the scarcity of annotated data often impedes the efficacy of prevailing methods. To address this issue, we introduce FuRPE, a novel framework that employs part-experts and an ingenious pseudo ground-truth selection scheme to derive high-quality pseudo labels. These labels, central to our approach, equip our network with the capability to efficiently learn from the available data. Integral to FuRPE is a unique exponential moving average training strategy and expert-derived feature distillation strategy. These novel elements of FuRPE not only serve to further refine the model but also to reduce potential biases that may arise from inaccuracies in pseudo labels, thereby optimizing the network's training process and enhancing the robustness of the model. We apply FuRPE to train both two-stage and fully convolutional single-stage full-body reconstruction networks. Our exhaustive experiments on numerous benchmark datasets illustrate a substantial performance boost over existing methods, underscoring FuRPE's potential to reshape the state-of-the-art in full-body reconstruction.
Paper Structure (19 sections, 18 equations, 3 figures, 6 tables)

This paper contains 19 sections, 18 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. 3D Human Pose Estimation
  4. 3D Human Body Reconstruction
  5. Full-body Reconstruction
  6. Method
  7. Overview
  8. Part Experts and Pseudo labels
  9. Pseudo Ground-truth Selection Scheme
  10. Training using Pseduo Labels
  11. Expert-Derived Feature Distillation Strategy
  12. Exponential Moving Average Training Strategy
  13. Experiment
  14. Implementation Details
  15. Dataset
  16. ...and 4 more sections

Figures (3)

  • Figure 1: (a) Traditional methods train on costly, scarce annotated data. (b) Our method utilizes affordable, high-quality pseudo labels from part-experts.
  • Figure 2: Pipeline of our work. (a) The training pipeline of using part-experts to generate supervision signals. (b) The training pipeline of Exponential Moving Average self-supervision.
  • Figure 3: Visualization results on EFH dataset.