Motion Attribution for Video Generation

TL;DR

This work tackles the challenge of understanding which training data shape temporal motion in video diffusion models. It introduces Motive, a motion-centric, gradient-based data attribution framework that uses motion-weighted masks to isolate dynamic regions and scale to large video datasets. By enabling motion-focused data curation and selective fine-tuning, Motive improves motion smoothness and dynamic degree, achieving strong human preference results and even surpassing full-dataset fine-tuning with only a fraction of data. The approach offers a scalable, interpretable tool for diagnosing motion artifacts and guiding data-driven improvements in video generation systems, with demonstrated generalization to larger models and multiple datasets.

Abstract

Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood. We present Motive (MOTIon attribution for Video gEneration), a motion-centric, gradient-based data attribution framework that scales to modern, large, high-quality video datasets and models. We use this to study which fine-tuning clips improve or degrade temporal dynamics. Motive isolates temporal dynamics from static appearance via motion-weighted loss masks, yielding efficient and scalable motion-specific influence computation. On text-to-video models, Motive identifies clips that strongly affect motion and guides data curation that improves temporal consistency and physical plausibility. With Motive-selected high-influence data, our method improves both motion smoothness and dynamic degree on VBench, achieving a 74.1% human preference win rate compared with the pretrained base model. To our knowledge, this is the first framework to attribute motion rather than visual appearance in video generative models and to use it to curate fine-tuning data.

Figures (9)

• Figure 1: Motive.Top. Motion-gradient computation (§\ref{['sec:method:2']}) has three steps: (1) detect motion with AllTracker; (2) compute motion-magnitude patches; (3) apply loss-space motion masks to focus gradients on dynamic regions. Bottom. Our method (§\ref{['sec:method:1']}) is made scalable via a single-sample variant with common randomness and a projection, computed for each pair of training and query data, aggregated (§\ref{['sec:method:3']}) for a final ranking, and eventually used to select fine-tuning subsets.
• Figure 2: Motion attribution examples.Top: Query clips showing float (left) and roll (right) motions. Middle: Top-ranked positive training samples identified by Motive with high influence scores. Bottom: Negative influence samples with minimal, camera-only motion, or cartoon-style content that conflict with target motions.
• Figure 3: Qualitative Comparisons. We compare four motion scenarios (compress, spin, slide, free fall) across the base model, random selection, and our method. Our approach yields more realistic motion dynamics. Supplementary videos are included.
• Figure 4: Projection dimension analysis. Spearman correlation between projected and full gradients shows rapid improvement with projection dimension, with $512$ providing a strong trade-off between accuracy and efficiency.
• Figure 5: Impact of Frame-Length Normalization on Motion Attribution. Comparison of top-ranked samples for floating motion query. Left: With proper frame-length normalization, top samples consistently exhibit floating motion (waves, floating objects, surfing). Right: Without normalization, rankings are biased by video length, resulting in no coherent patterns among top samples.