Deterministic-to-Stochastic Diverse Latent Feature Mapping for Human Motion Synthesis

TL;DR

This work tackles the challenge of generating diverse and plausible human motion sequences, addressing training instability and limited diversity in existing diffusion-based approaches. It introduces DSDFM, a two-stage framework consisting of (1) a motion reconstruction stage using VQVAE to learn a structured latent space, and (2) a diverse motion generation stage that combines a deterministic feature mapping (DerODE) with a stochastic diversification (DivSDE) to connect Gaussian latents to the motion latent space during sampling. The proposed method demonstrates state-of-the-art performance on unconditional and conditional motion synthesis across two standard datasets, with faster, more stable training and fewer parameters than many diffusion-based baselines. Ablation studies confirm the benefits of the DerODE and DivSDE components, and qualitative results illustrate diverse, coherent motion outputs suitable for animation and robotics applications.

Abstract

Human motion synthesis aims to generate plausible human motion sequences, which has raised widespread attention in computer animation. Recent score-based generative models (SGMs) have demonstrated impressive results on this task. However, their training process involves complex curvature trajectories, leading to unstable training process. In this paper, we propose a Deterministic-to-Stochastic Diverse Latent Feature Mapping (DSDFM) method for human motion synthesis. DSDFM consists of two stages. The first human motion reconstruction stage aims to learn the latent space distribution of human motions. The second diverse motion generation stage aims to build connections between the Gaussian distribution and the latent space distribution of human motions, thereby enhancing the diversity and accuracy of the generated human motions. This stage is achieved by the designed deterministic feature mapping procedure with DerODE and stochastic diverse output generation procedure with DivSDE.DSDFM is easy to train compared to previous SGMs-based methods and can enhance diversity without introducing additional training parameters.Through qualitative and quantitative experiments, DSDFM achieves state-of-the-art results surpassing the latest methods, validating its superiority in human motion synthesis.

Figures (8)

• Figure 1: Examples of the inference process for human motion synthesis. Our method aims to generate diverse and accurate human motion sequences through the designed generative model.
• Figure 2: The overview of the proposed method DSDFM. The red arrow denotes the first stage and the green arrow denotes the second stage of DSDFM.
• Figure 3: Qualitative results of DSDFM. We present the generated human motion sequences under different settings. The unconditional human motion sequences (top) are generated from the HumanAct12 dataset. The Action-to-Motion results (bottom) show the generated diverse motion sequences under the Sit and Run action labels, which are sampled from the HumanML3D dataset.
• Figure 4: Comparison of the inference time costs of our method on the HumanAct12 dataset. We calculate the ablation studies (left) and average inference time comparison with baselines (right). All tests are performed on the NVIDIA A100.
• Figure 5: Qualitative results of DSDFM. We present more generated unconditional human motion sequences.

Theorems & Definitions (2)

• proof
• proof