Text-driven Motion Generation: Overview, Challenges and Directions

TL;DR

Text-driven motion generation aims to synthesize realistic human motion from natural language prompts, enabling intuitive control of avatars without seed poses. The paper surveys two complementary viewpoints—architectural paradigms (VAE-based, diffusion-based, and hybrids) and motion representations (discrete vs continuous)—and details datasets, evaluation metrics, and current benchmarks. It highlights key challenges, including data scarcity, limited fine-grained articulation (hands/faces), and evaluation ambiguity, and discusses emerging directions toward large-scale motion models, controllability, and scene-aware generation. By consolidating existing work and outlining promising avenues, the survey aims to guide future research and practical deployments in language-grounded motion synthesis.

Abstract

Text-driven motion generation offers a powerful and intuitive way to create human movements directly from natural language. By removing the need for predefined motion inputs, it provides a flexible and accessible approach to controlling animated characters. This makes it especially useful in areas like virtual reality, gaming, human-computer interaction, and robotics. In this review, we first revisit the traditional perspective on motion synthesis, where models focused on predicting future poses from observed initial sequences, often conditioned on action labels. We then provide a comprehensive and structured survey of modern text-to-motion generation approaches, categorizing them from two complementary perspectives: (i) architectural, dividing methods into VAE-based, diffusion-based, and hybrid models; and (ii) motion representation, distinguishing between discrete and continuous motion generation strategies. In addition, we explore the most widely used datasets, evaluation methods, and recent benchmarks that have shaped progress in this area. With this survey, we aim to capture where the field currently stands, bring attention to its key challenges and limitations, and highlight promising directions for future exploration. We hope this work offers a valuable starting point for researchers and practitioners working to push the boundaries of language-driven human motion synthesis.