SnapMoGen: Human Motion Generation from Expressive Texts

TL;DR

SnapMoGen addresses the need for expressive, long-horizon text-to-motion generation by pairing 20K temporally continuous mocap clips (≈43.7–44 hours) with 122K highly expressive text descriptions. The authors introduce MoMask++, a unified text-to-token framework that uses multi-scale residual vector quantization with a shared codebook and a single masked transformer to produce all motion tokens from rich prompts, achieving state-of-the-art results on both HumanML3D and SnapMoGen. They demonstrate the practical utility of LLM-based prompt augmentation to translate casual prompts into SnapMoGen-style inputs and provide a comprehensive dataset and evaluation setup to advance long-term motion synthesis research. Overall, the work advances fine-grained controllability and generalization in text-conditioned motion generation and supplies a scalable benchmark for future exploration.

Abstract

Text-to-motion generation has experienced remarkable progress in recent years. However, current approaches remain limited to synthesizing motion from short or general text prompts, primarily due to dataset constraints. This limitation undermines fine-grained controllability and generalization to unseen prompts. In this paper, we introduce SnapMoGen, a new text-motion dataset featuring high-quality motion capture data paired with accurate, expressive textual annotations. The dataset comprises 20K motion clips totaling 44 hours, accompanied by 122K detailed textual descriptions averaging 48 words per description (vs. 12 words of HumanML3D). Importantly, these motion clips preserve original temporal continuity as they were in long sequences, facilitating research in long-term motion generation and blending. We also improve upon previous generative masked modeling approaches. Our model, MoMask++, transforms motion into multi-scale token sequences that better exploit the token capacity, and learns to generate all tokens using a single generative masked transformer. MoMask++ achieves state-of-the-art performance on both HumanML3D and SnapMoGen benchmarks. Additionally, we demonstrate the ability to process casual user prompts by employing an LLM to reformat inputs to align with the expressivity and narration style of SnapMoGen. Project webpage: https://snap-research.github.io/SnapMoGen/

Figures (8)

• Figure 1: Samples from SnapMoGen dataset. Our dataset features temporally continuous motion segments paired with highly expressive text annotations. Each motion clip is accompanied by six distinct textual descriptions. We show an LLM-augmented annotation for the first segment, and a human annotation for the second.
• Figure 2: Approach overview. (a) A multi-scale VQVAE encodes a motion sequence into $V+1$ discrete token sequences $(q^0, ..., q^V)$, where each sequence $q^v$ operates at a specific temporal resolution $h^v$. Their corresponding quantized features are upsampled to full resolution $\{\hat{f}_\uparrow^v\}_{v=0}^V$ and summed before being fed into the decoder. (b) A single masked transformer operates on tokens from all scales. Token sequences from a motion are concatenated along the temporal dimension and randomly masked with a variable rate. The transformer is trained to predict the masked tokens conditioned on text and the partially masked sequence. (c) We implement two methods for text conditioning: in-context learning and cross-attention.
• Figure 3: Illustration of token capacity in a pretrained traditional 6-layer, 480-tokenfull-scale RVQ guo2024momask compared to a 10-layer, 266-tokenmulti-scale RVQ. Starting from a zero-sequence, we incrementally add one quantized feature sequence for motion decoding and measure the reconstruction performance. The multi-scale VQ learns tokens more efficiently with meaningful semantics at each quantization layer.
• Figure 4: MoMask++ generated samples for SnapMoGen test prompts (#1,2) and a casual user prompt (#3).
• Figure 5: Decoding progress over iterations for different token scales.