Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds

Ziqiang Dang, Tianxing Fan, Boming Zhao, Xujie Shen, Lei Wang, Guofeng Zhang, Zhaopeng Cui

TL;DR

MoManifold is presented, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility.

Abstract

Incorporating temporal information effectively is important for accurate 3D human motion estimation and generation which have wide applications from human-computer interaction to AR/VR. In this paper, we present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Different from existing mathematical or VAE-based methods, our representation is designed based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility. Specifically, we propose novel decoupled joint acceleration manifolds to model human dynamics from existing limited motion data. Moreover, we introduce a novel optimization method using the manifold distance as guidance, which facilitates a variety of motion-related tasks. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks such as denoising real-world human mocap data, recovering human motion from partial 3D observations, mitigating jitters for SMPL-based pose estimators, and refining the results of motion in-betweening.

MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds

TL;DR

MoManifold is presented, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility.

Abstract

Incorporating temporal information effectively is important for accurate 3D human motion estimation and generation which have wide applications from human-computer interaction to AR/VR. In this paper, we present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Different from existing mathematical or VAE-based methods, our representation is designed based on the neural distance field, which makes human dynamics explicitly quantified to a score and thus can measure human motion plausibility. Specifically, we propose novel decoupled joint acceleration manifolds to model human dynamics from existing limited motion data. Moreover, we introduce a novel optimization method using the manifold distance as guidance, which facilitates a variety of motion-related tasks. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks such as denoising real-world human mocap data, recovering human motion from partial 3D observations, mitigating jitters for SMPL-based pose estimators, and refining the results of motion in-betweening.
Paper Structure (23 sections, 2 equations, 11 figures, 5 tables)

This paper contains 23 sections, 2 equations, 11 figures, 5 tables.

Table of Contents

  1. Implementation Details
  2. Weighted Design
  3. Data Preparation
  4. Optimization Details
  5. Experimental Evaluation Details
  6. Datasets
  7. Evaluation Metrics
  8. PVE of Legs and Feet
  9. Optimization Space of Rotation
  10. Extended Experiments
  11. Correlation Analysis
  12. Motion Fitting from 3D Observations
  13. Mitigating Jitters for SMPL-based Pose Estimators
  14. Motion In-betweening Refinement
  15. Motion Generation
  16. ...and 8 more sections

Figures (11)

  • Figure S1: SMPL Body Segmentation. The white box contains the segmentation of legs, feet and toe-bases.
  • Figure S3: Scatter Plots of Other Joints. Each blue point represents a motion segment.
  • Figure S4: VPoser-t Denoising Results.
  • Figure S5: HuMoR Accumulation of Errors.
  • Figure S7: Denoising Comparison. Body parts that are significantly different from the ground truth are marked in colored boxes. The results of VPoser-t are same with Figure \ref{['fig:vposer']}. For uniform motion of Pose-NDF, the legs begin to retract in the first frame, whereas at that time, the human should stand on the ground. Besides, the right arm and shoulders in the last two frames are obviously different from the ground truth. Since this is the beginning of the motion, there is no accumulation of errors for HuMoR. And our results are the closest to the ground truth.
  • ...and 6 more figures