Learning Generalizable Human Motion Generator with Reinforcement Learning

TL;DR

The paper tackles the generalization gap in text-driven human motion generation caused by scarce paired data and a many-to-many text-to-motion mapping. It proposes InstructMotion, an RL-based framework that treats text-to-motion generation as an MDP and fine-tunes a pre-trained autoregressive motion generator using PPO with a reward grounded in contrastive text/motion embeddings, enabling learning from both paired data and synthetic unpaired prompts. A data pipeline leverages large language models to generate novel motion descriptions and meta-motions, while the reward design balances text-to-motion and motion-to-motion alignment to improve generalization. Experiments on HumanML3D and KIT-ML show quantitative improvements in R-Precision and FID and qualitative gains in responding to novel prompts, with human evaluators favoring InstructMotion for unseen descriptions.

Abstract

Text-driven human motion generation, as one of the vital tasks in computer-aided content creation, has recently attracted increasing attention. While pioneering research has largely focused on improving numerical performance metrics on given datasets, practical applications reveal a common challenge: existing methods often overfit specific motion expressions in the training data, hindering their ability to generalize to novel descriptions like unseen combinations of motions. This limitation restricts their broader applicability. We argue that the aforementioned problem primarily arises from the scarcity of available motion-text pairs, given the many-to-many nature of text-driven motion generation. To tackle this problem, we formulate text-to-motion generation as a Markov decision process and present \textbf{InstructMotion}, which incorporate the trail and error paradigm in reinforcement learning for generalizable human motion generation. Leveraging contrastive pre-trained text and motion encoders, we delve into optimizing reward design to enable InstructMotion to operate effectively on both paired data, enhancing global semantic level text-motion alignment, and synthetic text-only data, facilitating better generalization to novel prompts without the need for ground-truth motion supervision. Extensive experiments on prevalent benchmarks and also our synthesized unpaired dataset demonstrate that the proposed InstructMotion achieves outstanding performance both quantitatively and qualitatively.

Figures (7)

• Figure 1: Examples generated from simple and compositional given textual descriptions. Our method significantly outperforms previous state-of-the-art method MoMask momask in terms of generalization capability to novel motion compositions. The compositional descriptions are generated with the aid of large language models, as discussed in Section \ref{['data_pipeline']}.
• Figure 2: The over pipeline of InstructMotion. Given a batch of textual prompts, the pre-trained autoregressive generator first produces the corresponding motion sequences, which is fed into the reward model along with the text prompts to assess the generation quality, yielding a matching score. The score, combined with the prediction logits and the the critic model output (omitted in the figure), is then organized by the PPO algorithm to optimize the generator in a inner training loop.
• Figure 3: An illustration of the LLM-assisted novel motion description generation pipeline.
• Figure 4: Qualitative comparisons with top-performing methods. Our InstructMotion exhibits enhanced generalization capability and accurately interpret novel combinations of motion instructions.
• Figure 5: