A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques

TL;DR

<3-5 sentence high-level summary>

Abstract

Body and face motion play an integral role in communication. They convey crucial information on the participants. Advances in generative modeling and multi-modal learning have enabled motion generation from signals such as speech, conversational context and visual cues. However, generating expressive and coherent face and body dynamics remains challenging due to the complex interplay of verbal / non-verbal cues and individual personality traits. This survey reviews body and face motion generation, covering core concepts, representations techniques, generative approaches, datasets and evaluation metrics. We highlight future directions to enhance the realism, coherence and expressiveness of avatars in dyadic settings. To the best of our knowledge, this work is the first comprehensive review to cover both body and face motion. Detailed resources are listed on https://lownish23csz0010.github.io/mogen/.

Figures (6)

• Figure 1: Overview of Generic Motion Generation Pipeline of existing SOTAs. Given the input from the respective modalities, the methods generate desired body or face motion using appropriate representation techniques.
• Figure 2: (Top) Given the body image, the body pose and geometry are reconstructed using body representation techniques. Source: SMPL-X:2019 (Bottom) Given face images, the face is parameterized and reconstructed using 3DMM frameworks, effectively capturing the pose and expression. Source: retsinas20243d
• Figure 3: Visualization of existing motion generation frameworks. For better view, kindly visit the https://lownish23csz0010.github.io/mogen/
• Figure 4: Overview of Performance Evaluation Metrics as represented in Table \ref{['tab:metrics_list']}
• Figure 5: Roadmap of Motion Generation Techniques