Back to Basics: Motion Representation Matters for Human Motion Generation Using Diffusion Model

TL;DR

This work investigates how the choice of motion representation and loss function affects diffusion-based human motion generation. By evaluating six representations within a v-parameter diffusion framework (vMDM) across multiple datasets, the authors quantify impacts on fidelity, diversity, and training efficiency, and compare to standard MDM baselines. They demonstrate that a position-based representation (JP), when combined with the v loss and geometric terms, yields superior quantitative performance and faster training, while rotation-based representations offer smoother temporality but can introduce artifacts. The findings provide practical guidance for designing motion diffusion pipelines and highlight the trade-offs between representation, loss design, and computational cost.

Abstract

Diffusion models have emerged as a widely utilized and successful methodology in human motion synthesis. Task-oriented diffusion models have significantly advanced action-to-motion, text-to-motion, and audio-to-motion applications. In this paper, we investigate fundamental questions regarding motion representations and loss functions in a controlled study, and we enumerate the impacts of various decisions in the workflow of the generative motion diffusion model. To answer these questions, we conduct empirical studies based on a proxy motion diffusion model (MDM). We apply v loss as the prediction objective on MDM (vMDM), where v is the weighted sum of motion data and noise. We aim to enhance the understanding of latent data distributions and provide a foundation for improving the state of conditional motion diffusion models. First, we evaluate the six common motion representations in the literature and compare their performance in terms of quality and diversity metrics. Second, we compare the training time under various configurations to shed light on how to speed up the training process of motion diffusion models. Finally, we also conduct evaluation analysis on a large motion dataset. The results of our experiments indicate clear performance differences across motion representations in diverse datasets. Our results also demonstrate the impacts of distinct configurations on model training and suggest the importance and effectiveness of these decisions on the outcomes of motion diffusion models.

Figures (9)

• Figure 1: To test the importance of various motion representations, the framework of $v$MDM consists of two stages: training and inference. In the training stage, we first encode the clean motion sequences within $N$ frames to six motion representations (JP, RP6JR, RPQJR, RPAJR, RPEJR, RPMJR). Second, we procedurally add noise to the processed motion data using a forward diffusion module and get noisy motion data after $T$ diffusion time steps. Third, we train a denoiser using a Transformer architecture, and our objective prediction is $v$ parameterization. Based on $v$ prediction, we can recover motion data with a motion representation consistent with the input provided to the denoiser. During the inference stage, our input to the Transformer denoiser is pure Gaussian noise. We then apply the reverse diffusion module and a Gaussian filter to generate motion sequences.
• Figure 2: The average smoothness scores of six motion representations across the motion clips of the HumanAct12 motion dataset.
• Figure 3: Feature heatmaps of a single motion clip with diverse motion representations based on the HumanAct12 dataset.
• Figure 4: Qualitative comparisons of generated motion sequences with various motion representations from $v$MDM based on HumanAct12 motion dataset. For clear visualization, we separate the poses at equal intervals across the frames (10-frame interval). The lighter the color, the smaller the frame index, and vice versa.
• Figure 5: Qualitative comparisons of generated motion sequences with various motion representations from MDM based on HumanAct12 motion dataset. For clear visualization, we separate the poses at equal intervals across the frames. The lighter the colour, the smaller the frame index, and vice versa. In the yellow box, the poses remain almost stationary.