Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers

Ahmet Berke Gokmen, Yigit Ekin, Bahri Batuhan Bilecen, Aysegul Dundar

TL;DR

RoPECraft tackles the problem of controlling spatiotemporal motion in diffusion-video generation without backbone training. It introduces motion-augmented RoPE, warping RoPE with optical-flow displacements, followed by flow-matching optimization and Fourier-phase regularization to ensure temporal coherence and reduce artifacts. The approach achieves state-of-the-art performance on benchmarks, with favorable MF, CD-FVD, CLIP, and the new Fréchet Trajectory Distance (FTD) metrics, while being more computationally efficient than tuning-based methods. This training-free method enables practical motion transfer and has potential for controllable video editing and robust motion transfer under domain shifts.

Abstract

We propose RoPECraft, a training-free video motion transfer method for diffusion transformers that operates solely by modifying their rotary positional embeddings (RoPE). We first extract dense optical flow from a reference video, and utilize the resulting motion offsets to warp the complex-exponential tensors of RoPE, effectively encoding motion into the generation process. These embeddings are then further optimized during denoising time steps via trajectory alignment between the predicted and target velocities using a flow-matching objective. To keep the output faithful to the text prompt and prevent duplicate generations, we incorporate a regularization term based on the phase components of the reference video's Fourier transform, projecting the phase angles onto a smooth manifold to suppress high-frequency artifacts. Experiments on benchmarks reveal that RoPECraft outperforms all recently published methods, both qualitatively and quantitatively.

RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers

TL;DR

RoPECraft tackles the problem of controlling spatiotemporal motion in diffusion-video generation without backbone training. It introduces motion-augmented RoPE, warping RoPE with optical-flow displacements, followed by flow-matching optimization and Fourier-phase regularization to ensure temporal coherence and reduce artifacts. The approach achieves state-of-the-art performance on benchmarks, with favorable MF, CD-FVD, CLIP, and the new Fréchet Trajectory Distance (FTD) metrics, while being more computationally efficient than tuning-based methods. This training-free method enables practical motion transfer and has potential for controllable video editing and robust motion transfer under domain shifts.

Abstract

We propose RoPECraft, a training-free video motion transfer method for diffusion transformers that operates solely by modifying their rotary positional embeddings (RoPE). We first extract dense optical flow from a reference video, and utilize the resulting motion offsets to warp the complex-exponential tensors of RoPE, effectively encoding motion into the generation process. These embeddings are then further optimized during denoising time steps via trajectory alignment between the predicted and target velocities using a flow-matching objective. To keep the output faithful to the text prompt and prevent duplicate generations, we incorporate a regularization term based on the phase components of the reference video's Fourier transform, projecting the phase angles onto a smooth manifold to suppress high-frequency artifacts. Experiments on benchmarks reveal that RoPECraft outperforms all recently published methods, both qualitatively and quantitatively.
Paper Structure (26 sections, 5 equations, 15 figures, 6 tables, 2 algorithms)

This paper contains 26 sections, 5 equations, 15 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Text-to-video models.
  4. Motion transfer.
  5. Positional embeddings in vision transformers.
  6. Preliminaries
  7. Flow matching
  8. Rotary position embeddings (RoPE)
  9. Methodology
  10. Motion-augmented RoPE
  11. Optimization with flow-matching
  12. Phase constraints
  13. Experimental Results
  14. Metrics and baselines
  15. Fréchet Trajectory Distance
  16. ...and 11 more sections

Figures (15)

  • Figure 1: Our method successfully transfers the motion from reference videos.
  • Figure 2: Latent warping burgert2025gowiththeflow without an expensive fine-tuning of the DiT fails (Column 2), and latent optimization is not adequate to recover the domain shift (Column 3). Our approach keeps the latent space intact, and performs successful motion transfer, all without model re-training (Column 4).
  • Figure 3: Visual description of our proposed pipeline inference and RoPE optimization approach.
  • Figure 4: Default 1D RoPE, expanded to 3D
  • Figure 5: Qualitative results of motion-augmented RoPE described in \ref{['algo:warped_rope']}.
  • ...and 10 more figures