Deep Compositional Phase Diffusion for Long Motion Sequence Generation

TL;DR

The paper tackles long-term compositional motion generation by introducing Compositional Phase Diffusion, which operates in a unified latent phase space produced by ACT-PAE and refined by SPDM and TPDM. SPDM injects semantic guidance via CLIP-conditioned denoising, while TPDM enforces bidirectional transitional coherence from neighboring segments, enabling parallel denoising across multiple motion clips. The framework demonstrates strong performance in compositional generation, motion inbetweening, and long-term generation on the BABEL-TEACH dataset, with ablations validating the importance of frame-level tokens and the phase-mixing strategy. This approach offers scalable, flexible motion synthesis with improved transition realism and semantic fidelity, with potential extensions to more advanced diffusion paradigms and adaptive phase mixing.

Abstract

Recent research on motion generation has shown significant progress in generating semantically aligned motion with singular semantics. However, when employing these models to create composite sequences containing multiple semantically generated motion clips, they often struggle to preserve the continuity of motion dynamics at the transition boundaries between clips, resulting in awkward transitions and abrupt artifacts. To address these challenges, we present Compositional Phase Diffusion, which leverages the Semantic Phase Diffusion Module (SPDM) and Transitional Phase Diffusion Module (TPDM) to progressively incorporate semantic guidance and phase details from adjacent motion clips into the diffusion process. Specifically, SPDM and TPDM operate within the latent motion frequency domain established by the pre-trained Action-Centric Motion Phase Autoencoder (ACT-PAE). This allows them to learn semantically important and transition-aware phase information from variable-length motion clips during training. Experimental results demonstrate the competitive performance of our proposed framework in generating compositional motion sequences that align semantically with the input conditions, while preserving phase transitional continuity between preceding and succeeding motion clips. Additionally, motion inbetweening task is made possible by keeping the phase parameter of the input motion sequences fixed throughout the diffusion process, showcasing the potential for extending the proposed framework to accommodate various application scenarios. Codes are available at https://github.com/asdryau/TransPhase.

Figures (7)

• Figure 1: Our Compositional Phase Diffusion framework produces high-quality composite motion sequences with smooth transitions and semantic alignment. (a) Compositional generation involves synthesizing multiple motion segments of varying lengths simultaneously, ensuring smooth transitions between segments. (b) Motion inbetweening allows users to select segments (in blue and yellow) and create conditional or unconditional bridging motions. (c) Long-term motion generation is achieved by scaling the framework with additional modules, enabling the parallel denoising of a larger number of motion segments. The rainbow color indicates time progression.
• Figure 2: Module detail of TPDM and SPDM. (a) The TPDM uses the clean phase latents from the adjacent segment $\mathbf{P}_\mathbf{p}^0$ to denoise the current motion phase $\mathbf{P}_\mathbf{t}^k$, making the current denoising motion align with the motion dynamics of the adjacent motion segment. (b) The SPDM utilizes the text embedding $C$ from CLIP to denoise the motion phase parameters of the current denoising motion.
• Figure 3: Illustration of the phase diffusion pipeline for the compositional motion generation task. SPDMs and TPDMs guide the denoising of motion segments through semantic information and the phase dynamics information from adjacent motions, respectively. The denoised results are combined via phase mixing and either diffused back to step $k-1$ or fed into the ACT-PAE decoder at the final step to produce motion segments, which are then linearly blended to create a long sequence.
• Figure 4: The phase diffusion pipeline for the motion inbetweening tasks. The inbetweening motion $\mathbf{X_i}$ along with two transitional motions $\mathbf{X_{t_1}}$ and $\mathbf{X_{t_2}}$ are simultaneously generated. These motions are subsequently linear blended to form the final inbetweening output.
• Figure 5: Compositional motion pair result visualization for [walk(2.4s), sit down(3.6s)]. The motion frames are colored from red to purple in a rainbow gradient to represent the progression of time. Note that priorMDM exhibits an unrealistic, sudden turn during the sit-down action, which is reflected in its low FID score. TEACH's result includes footing skating in the walk motion. In contrast, our framework generates a fluid walking motion that transitions smoothly into a sit-down action.