Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation

TL;DR

Latent Motion Reasoning (LMR) is proposed, a novel Dual-Granularity Tokenizer that disentangles motion into two distinct manifolds: a compressed, semantically rich Reasoning Latent for planning global topology, and a high-frequency Execution Latent for preserving physical fidelity.

Abstract

Current state-of-the-art paradigms predominantly treat Text-to-Motion (T2M) generation as a direct translation problem, mapping symbolic language directly to continuous poses. While effective for simple actions, this System 1 approach faces a fundamental theoretical bottleneck we identify as the Semantic-Kinematic Impedance Mismatch: the inherent difficulty of grounding semantically dense, discrete linguistic intent into kinematically dense, high-frequency motion data in a single shot. In this paper, we argue that the solution lies in an architectural shift towards Latent System 2 Reasoning. Drawing inspiration from Hierarchical Motor Control in cognitive science, we propose Latent Motion Reasoning (LMR) that reformulates generation as a two-stage Think-then-Act decision process. Central to LMR is a novel Dual-Granularity Tokenizer that disentangles motion into two distinct manifolds: a compressed, semantically rich Reasoning Latent for planning global topology, and a high-frequency Execution Latent for preserving physical fidelity. By forcing the model to autoregressively reason (plan the coarse trajectory) before it moves (instantiates the frames), we effectively bridge the ineffability gap between language and physics. We demonstrate LMR's versatility by implementing it for two representative baselines: T2M-GPT (discrete) and MotionStreamer (continuous). Extensive experiments show that LMR yields non-trivial improvements in both semantic alignment and physical plausibility, validating that the optimal substrate for motion planning is not natural language, but a learned, motion-aligned concept space. Codes and demos can be found in \hyperlink{https://chenhaoqcdyq.github.io/LMR/}{https://chenhaoqcdyq.github.io/LMR/}

Figures (10)

• Figure 1: Architectural Comparison and the Central Challenge of Text-to-Motion (T2M) Generation. Existing methods struggle to bridge the gap between abstract language and continuous motion. (a) Direct GenerationT2MMotionStreamer and (b) Direct Generation with FeedbackMotionRLyue2025rl (collectively, System 1) are severely limited by the Semantic-Kinematic Impedance, where the generator must simultaneously plan global trajectory and guarantee local physical fidelity, resulting in a single-shot, over-burdened process and outputs that lack physical grounding. (c) The Language CoT ParadigmMotionR1 introduces high-level reasoning (System 2), but the textual chain-of-thought is a Low-Bandwidth Bottleneck (Funnel), losing the high-frequency information necessary for realistic dynamics—a limitation known as the Ineffability of Language. (d) Our Latent Motion Reasoning (LMR) overcomes these limitations by operating entirely within the Dual-Granularity Latent Space. This "Think-then-Act" architectural shift disentangles planning from execution, allowing the model to autoregressively reason over a compressed trajectory before instantiating the physically plausible motion, effectively harmonizing semantic intent with kinematic demands.
• Figure 2: Visualization of the Manifold Orthogonality between Semantic Alignment and Kinematic Fidelity. We employ t-SNE to project motion representations learned under three distinct objectives: (Left) Reconstruction-only ($\mathcal{L}_{rec}$), (Middle) Alignment-only ($\mathcal{L}_{align}$), and (Right) Our LMR Framework. The Top Row colors samples by high-level semantic categories (e.g., Interactions, Object Handling), while the Bottom Row colors them by specific kinematic sequences, which is ineffability, thus denoted as Action X. Observation: A fundamental trade-off is observed in single-objective baselines. The Reconstruction objective yields tight kinematic clusters (Left-Bottom) but results in semantically entangled manifolds (Left-Top). Conversely, the Alignment objective creates clear semantic boundaries (Middle-Top) but causes kinematic blurring (Middle-Bottom), losing fine-grained physical distinctiveness. (Right) Our LMR, via its Dual-Granularity Tokenizer, successfully reconciles this conflict, maintaining clear separability in both semantic intent and kinematic execution spaces.
• Figure 3: Analysis of Semantic Density and Temporal Resolution. (Left) The Reconstruction-Generation Trade-off: We plot reconstruction error (MPJPE, Blue) and generation quality (FID, Green) across varying down-sampling ratios. A clear trade-off is observed: while high-frequency tokens ($1/1$) yield the best reconstruction, they degrade generation quality. The optimal balance is found at a $1/4$ ratio (Red Star). Crucially, the "Same Token Length" comparison (Triangles vs. Circles with a temporal down-sampling ratio of $\frac{1}{1}$) reveals that a long motion sequence compressed by $4\times$ (Hollow Triangle) generates significantly better motion than a short, uncompressed sequence of the same token count ( $\frac{1}{1}$ Circles). This confirms that Semantic Density, rather than sequence length, is the governing factor for generation quality. (Middle & Right) Information Sparsity Probe: We evaluate semantic robustness by testing the retrieval performance using token sequences with randomly dropped tokens. The high-frequency representation (Middle, $1\times$) exhibits higher redundancy (slower decay in Cosine similarity and Top-1 Accuracy) compared to the compressed representation (Right, $4\times$). This indicates that semantic information in raw motion is highly sparse and diluted, creating a "contextual sprawl" that hinders effective autoregressive modeling.
• Figure 4: Overview of the proposed Latent Motion Reasoning (LMR) framework. The framework consists of two phases: (Right) Dual-Granularity (DG) Tokenizer: We explicitly disentangle motion representations into two manifolds: a compressed Reasoning Latent (Yellow), which is aligned with text embeddings to capture high-level semantic intent, and a high-frequency Execution Latent (Blue), which preserves low-level kinematic fidelity for reconstruction. (Left) LMR-Generator: We reformulate T2M as a hierarchical "Think-then-Act" generation process. Conditioned on the text prompt, the model first autoregressively synthesizes the coarse-grained reasoning tokens to establish the global motion topology (Thinking Phase). These tokens then serve as a stable semantic condition to guide the subsequent generation of fine-grained execution tokens (Acting Phase) via either Categorical or Diffusive sampling.
• Figure 5: Qualitative comparison with state-of-the-art methods under the discrete motion representation setting.