Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation

Xingzu Zhan, Chen Xie, Honghang Chen, Yixun Lin, Xiaochun Mai

TL;DR

T2M Mamba addresses drift and paraphrase sensitivity in long-horizon text-to-motion generation by coupling motion periodicity with keyframe saliency and by robustly aligning text and motion embeddings. It introduces Periodicity-Saliency Aware Mamba, which injects keyframe weights and rhythm phase into the Mamba backbone, and PDCAM, a differential cross-modal attention module that leverages phase-aware cues for stable alignment. Key technical contributions include an enhanced Density Peaks Clustering for adaptive keyframe detection and FFT-accelerated autocorrelation for dominant period estimation, both integrated with minimal overhead. Empirical results on HumanML3D and KIT-ML show state-of-the-art FID$=0.068$, strong R-precision, and demonstrated paraphrase robustness, highlighting practical gains for long-horizon, text-driven motion synthesis.

Abstract

Text-to-motion generation, which converts motion language descriptions into coherent 3D human motion sequences, has attracted increasing attention in fields, such as avatar animation and humanoid robotic interaction. Though existing models have achieved significant fidelity, they still suffer from two core limitations: (i) They treat motion periodicity and keyframe saliency as independent factors, overlooking their coupling and causing generation drift in long sequences. (ii) They are fragile to semantically equivalent paraphrases, where minor synonym substitutions distort textual embeddings, propagating through the decoder and producing unstable or erroneous motions. In this work, we propose T2M Mamba to address these limitations by (i) proposing Periodicity-Saliency Aware Mamba, which utilizes novel algorithms for keyframe weight estimation via enhanced Density Peaks Clustering and motion periodicity estimation via FFT-accelerated autocorrelation to capture coupled dynamics with minimal computational overhead, and (ii) constructing a Periodic Differential Cross-modal Alignment Module (PDCAM) to enhance robust alignment of textual and motion embeddings. Extensive experiments on HumanML3D and KIT-ML datasets have been conducted, confirming the effectiveness of our approach, achieving an FID of 0.068 and consistent gains on all other metrics.

T2M Mamba: Motion Periodicity-Saliency Coupling Approach for Stable Text-Driven Motion Generation

TL;DR

T2M Mamba addresses drift and paraphrase sensitivity in long-horizon text-to-motion generation by coupling motion periodicity with keyframe saliency and by robustly aligning text and motion embeddings. It introduces Periodicity-Saliency Aware Mamba, which injects keyframe weights and rhythm phase into the Mamba backbone, and PDCAM, a differential cross-modal attention module that leverages phase-aware cues for stable alignment. Key technical contributions include an enhanced Density Peaks Clustering for adaptive keyframe detection and FFT-accelerated autocorrelation for dominant period estimation, both integrated with minimal overhead. Empirical results on HumanML3D and KIT-ML show state-of-the-art FID, strong R-precision, and demonstrated paraphrase robustness, highlighting practical gains for long-horizon, text-driven motion synthesis.

Abstract

Text-to-motion generation, which converts motion language descriptions into coherent 3D human motion sequences, has attracted increasing attention in fields, such as avatar animation and humanoid robotic interaction. Though existing models have achieved significant fidelity, they still suffer from two core limitations: (i) They treat motion periodicity and keyframe saliency as independent factors, overlooking their coupling and causing generation drift in long sequences. (ii) They are fragile to semantically equivalent paraphrases, where minor synonym substitutions distort textual embeddings, propagating through the decoder and producing unstable or erroneous motions. In this work, we propose T2M Mamba to address these limitations by (i) proposing Periodicity-Saliency Aware Mamba, which utilizes novel algorithms for keyframe weight estimation via enhanced Density Peaks Clustering and motion periodicity estimation via FFT-accelerated autocorrelation to capture coupled dynamics with minimal computational overhead, and (ii) constructing a Periodic Differential Cross-modal Alignment Module (PDCAM) to enhance robust alignment of textual and motion embeddings. Extensive experiments on HumanML3D and KIT-ML datasets have been conducted, confirming the effectiveness of our approach, achieving an FID of 0.068 and consistent gains on all other metrics.
Paper Structure (22 sections, 13 equations, 5 figures, 4 tables)

This paper contains 22 sections, 13 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text-to-Motion Generation
  4. State Space Models
  5. Methodology
  6. Overview of T2M Mamba
  7. Motion Periodicity-Saliency Coupling
  8. keyframe weight estimation
  9. Motion Periodicity Estimation
  10. Periodicity-Saliency Aware Mamba
  11. Periodic Differential Cross‑modal Alignment Module
  12. Experiments
  13. Datasets
  14. Datasets
  15. Comparison with State-of-the-arts
  16. ...and 7 more sections

Figures (5)

  • Figure 1: The overview of the proposed T2M Mamba.(a) T2M Mamba. Our T2M Mamba consisting of N basic blocks aims to predict clean motion sequence (b) Inference Process. Starting from Gaussian noise, the model iteratively denoises to generate a clean motion sequence $M^0$ semantically aligned with the input text prompt.
  • Figure 2: Illustration of our Periodicity-Saliency Aware Mamba. $\odot$ denotes dot product.
  • Figure 3: Illustration of the proposed PDCAM pipeline.
  • Figure 4: Qualitative comparison of prominent state-of-the-art methods.
  • Figure 5: Visual comparison of T2M Mamba under subtle semantic variations