Free-T2M: Robust Text-to-Motion Generation for Humanoid Robots via Frequency-Domain

TL;DR

This work rethinks text-to-motion generation for humanoid robots by analyzing diffusion denoising in the frequency domain, revealing a coarse-to-fine, hierarchical planning process. It introduces Free-T2M, which couples a Time-Aware Self-Temporal Attention mechanism with full-spectrum DCT-space auxiliary supervision to enforce semantic planning in low frequencies and precise high-frequency execution. The approach yields state-of-the-art or competitive performance on HumanML3D and KIT-ML benchmarks, with substantial gains in FID and R-Precision and robust handling of text perturbations. By explicitly aligning frequency components across diffusion steps, Free-T2M improves semantic fidelity, temporal coherence, and practical reliability for natural language control of humanoid robots.

Abstract

Enabling humanoid robots to synthesize complex, physically coherent motions from natural language commands is a cornerstone of autonomous robotics and human-robot interaction. While diffusion models have shown promise in this text-to-motion (T2M) task, they often generate semantically flawed or unstable motions, limiting their applicability to real-world robots. This paper reframes the T2M problem from a frequency-domain perspective, revealing that the generative process mirrors a hierarchical control paradigm. We identify two critical phases: a semantic planning stage, where low-frequency components establish the global motion trajectory, and a fine-grained execution stage, where high-frequency details refine the movement. To address the distinct challenges of each phase, we introduce Frequency enhanced text-to-motion (Free-T2M), a framework incorporating stage-specific frequency-domain consistency alignment. We design a frequency-domain temporal-adaptive module to modulate the alignment effects of different frequency bands. These designs enforce robustness in the foundational semantic plan and enhance the accuracy of detailed execution. Extensive experiments show our method dramatically improves motion quality and semantic correctness. Notably, when applied to the StableMoFusion baseline, Free-T2M reduces the FID from 0.152 to 0.060, establishing a new state-of-the-art within diffusion architectures. These findings underscore the critical role of frequency-domain insights for generating robust and reliable motions, paving the way for more intuitive natural language control of robots.

Figures (5)

• Figure 1: T2M Denoising as Hierarchical Motion Planning: Early stages serve as semantic planning for low-frequency recovery, later stages refine high-frequency details. As shown in the lower part of the figure, failures in low-frequency semantics lead to cascading errors, while poor high-frequency refinement yields incoherent motions. Our frequency-adaptive alignment enhances planning, improving semantic fidelity and naturalness.
• Figure 2: Overview of the Free-T2M Framework for Robotic Motion Synthesis. Free-T2M employs a UNet architecture featuring two core components. frequency-domain temporal-adaptive module (FTA) conditions the temporal attention on the diffusion timestep $t$, enabling an adaptive focus that shifts from coarse, task-level planning (low frequencies) in early denoising stages to fine-grained motor control (high frequencies) in later stages. An auxiliary head, supervised in the full Discrete Cosine Transform (DCT) space, ensures multi-scale temporal consistency, which is critical for generating physically plausible and executable robotic motions.
• Figure 3: Visualization results. The white arrows represent motion trajectories. In several examples, the baseline model produces errors in low-frequency information, leading to incorrect trajectories and ultimately affecting the accuracy of the motion. Our model demonstrates superior semantic consistency and finer-grained local actions compared to the baseline.
• Figure 4: Comparative performance of ANT (StableMoFusion) versus other baseline methods based on human evaluation. The left panel shows the accuracy (%) obtained during manual assessments. The right panel presents the preference score, computed as the proportion of times each method’s generated motion was selected as the most preferred among all four methods in each evaluation.
• Figure 5: FTA attention across timesteps. Row-normalized heatmap of frequency weights computed by applying a DCT to UNet outputs (averaged over joints) with log-compression and re-binned frequency axis. Early denoising steps (large $t$) concentrate on low-frequency bands (left); as $t$ decreases, energy shifts toward mid/high frequencies (right), evidencing the coarse-to-fine generation behavior and validating the time-adaptive design of FTA and our frequency-alignment objective.

Theorems & Definitions (1)

• Theorem 1