FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editing

TL;DR

This work tackles the data bottleneck hindering generalizable First-Frame Propagation (FFP) video editing by introducing FFP-300K, a large-scale 720p, 81-frame dataset with separate local-editing and global-stylization tracks. Building on this dataset, it proposes FreeProp, a guidance-free FFP framework that combines Adaptive Spatio-Temporal RoPE (AST-RoPE) with a self-distillation strategy via Identity Propagation to maintain temporal stability while preserving first-frame appearance. The approach optimizes with a flow-matching objective plus motion and MMD-based distillation terms, formalized as $\\mathcal{L}=\\mathcal{L}_{FM}+\\lambda_{motion}\\mathcal{L}_{motion}+\\lambda_{MMD}\\mathcal{L}_{MMD}$. On EditVerseBench, FreeProp achieves state-of-the-art fidelity and temporal coherence, outperforming both academic and commercial baselines by meaningful margins and enabling practical, fully guidance-free video editing.

Abstract

First-Frame Propagation (FFP) offers a promising paradigm for controllable video editing, but existing methods are hampered by a reliance on cumbersome run-time guidance. We identify the root cause of this limitation as the inadequacy of current training datasets, which are often too short, low-resolution, and lack the task diversity required to teach robust temporal priors. To address this foundational data gap, we first introduce FFP-300K, a new large-scale dataset comprising 300K high-fidelity video pairs at 720p resolution and 81 frames in length, constructed via a principled two-track pipeline for diverse local and global edits. Building on this dataset, we propose a novel framework designed for true guidance-free FFP that resolves the critical tension between maintaining first-frame appearance and preserving source video motion. Architecturally, we introduce Adaptive Spatio-Temporal RoPE (AST-RoPE), which dynamically remaps positional encodings to disentangle appearance and motion references. At the objective level, we employ a self-distillation strategy where an identity propagation task acts as a powerful regularizer, ensuring long-term temporal stability and preventing semantic drift. Comprehensive experiments on the EditVerseBench benchmark demonstrate that our method significantly outperforming existing academic and commercial models by receiving about 0.2 PickScore and 0.3 VLM score improvement against these competitors.

Figures (12)

• Figure 1: Overview of our Data Construction Pipeline. Our pipeline has two parallel tracks. Left: The local editing track performs object Swap and Removal. For swapping, we use target objects and captions from the source video to generate edits with erosion masks, followed by a quality filtering step. For removal, captions are constructed and paired with bounding-box masks to generate the edited videos. Notably, filtered samples are used to refine our VACE vace model, which then regenerates the entire removal subset for higher quality (Sec. \ref{['sec:localediting']}). Right: The global stylization track first generates source videos from images using Wan-I2V. It then combines these source videos, style reference images, and corresponding depth videos to produce high-fidelity stylized results (Sec. \ref{['sec:globalstylization']}).
• Figure 2: Overview of training paradigm. Left: The source video and edited frame are encoded. The source latent informs our AST-RoPE module for adaptive spatio-temporal scaling. Right: The target video is processed identically to extract a latent DiT embedding, which is used to align the generation process.
• Figure 3: Qualitative comparison. We Choose top three method in quantitative comparison to compare with our visual results across four representative video editing tasks. Red boxes highlight the unreasonable generated contents. The gray placeholder denotes these methods cannot generate such long videos. Our method generally enjoys better editing fidelity, temporal consistency and visual quality.
• Figure 4: Word cloud of edited objects of the local editing subst of FFP-300K.
• Figure 5: Scene distribution of the local editing subset of FFP-300K