Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two-Person Interaction Augmentation with Skeleton Priors

Baiyi Li, Edmond S. L. Ho, Hubert P. H. Shum, He Wang

TL;DR

This work proposes a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies.

Abstract

Close and continuous interaction with rich contacts is a crucial aspect of human activities (e.g. hugging, dancing) and of interest in many domains like activity recognition, motion prediction, character animation, etc. However, acquiring such skeletal motion is challenging. While direct motion capture is expensive and slow, motion editing/generation is also non-trivial, as complex contact patterns with topological and geometric constraints have to be retained. To this end, we propose a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies. Our system can learn effectively from a relatively small amount of data and generalize to drastically different skeleton sizes. Through exhaustive evaluation and comparison, we show it can generate high-quality motions, has strong generalizability and outperforms traditional optimization-based methods and alternative deep learning solutions.

Two-Person Interaction Augmentation with Skeleton Priors

TL;DR

This work proposes a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies.

Abstract

Close and continuous interaction with rich contacts is a crucial aspect of human activities (e.g. hugging, dancing) and of interest in many domains like activity recognition, motion prediction, character animation, etc. However, acquiring such skeletal motion is challenging. While direct motion capture is expensive and slow, motion editing/generation is also non-trivial, as complex contact patterns with topological and geometric constraints have to be retained. To this end, we propose a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies. Our system can learn effectively from a relatively small amount of data and generalize to drastically different skeleton sizes. Through exhaustive evaluation and comparison, we show it can generate high-quality motions, has strong generalizability and outperforms traditional optimization-based methods and alternative deep learning solutions.
Paper Structure (31 sections, 10 equations, 12 figures, 26 tables)

This paper contains 31 sections, 10 equations, 12 figures, 26 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep Learning for Skeletal Motion
  4. Motion Retargeting
  5. Interaction
  6. Methodology
  7. A New Factorization of Interaction Motions
  8. Network Architecture
  9. Loss functions
  10. A New Interaction Dataset
  11. Experiments
  12. Tasks, Metrics and Generalization Settings
  13. Evaluation
  14. Retargeting and Generation
  15. Extrapolating to Large Unseen Scales
  16. ...and 16 more sections

Figures (12)

  • Figure 1: Overview of our model. The key components include Spatial-temporal Graph Convolution Networks (ST-GCN), Multi-layer perceptrons (MLP) and G-GRU networks. Details are in the supplementary material.
  • Figure 2: The architecture of ST-GCN1 and G-GRU1. More details are in the supplementary material.
  • Figure 3: The architecture of ST-GCN2, ST-GCN3 and G-GRU2. More details are in the supplementary material.
  • Figure 4: In the original Judo motion (top), the red character is augmented for a bigger body (middle) and a smaller body (bottom), while retaining the key features of the interaction semantics. The black boxes in column a highlight how the "Judo holding" semantics, i.e., the red character holding the blue one, are adapted. The black boxes in column b show a similar example.
  • Figure 5: Comparison between ground-truth (top) and cross-scale-interaction (bottom). The skeleton of the red character is changed. Both of them are Back-flip on scale 0.85.
  • ...and 7 more figures