UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing

TL;DR

UniEdit introduces a tuning-free framework for joint video motion and appearance editing by leveraging a pre-trained text-to-video generator in an inversion-then-generation setup. It employs two auxiliary branches—a reconstruction branch for content preservation and a motion-reference branch for motion guidance—injecting their features into the main editing path via temporal and spatial self-attention, supplemented by mask-guided coordination and spatial-structure control. The approach achieves state-of-the-art results on both motion and appearance editing tasks and supports zero-shot text-image-to-video generation through TI2V capabilities. This work significantly advances practical, generalizable video editing by eliminating the need for fine-tuning while maintaining content integrity and temporal consistency.

Abstract

Recent advances in text-guided video editing have showcased promising results in appearance editing (e.g., stylization). However, video motion editing in the temporal dimension (e.g., from eating to waving), which distinguishes video editing from image editing, is underexplored. In this work, we present UniEdit, a tuning-free framework that supports both video motion and appearance editing by harnessing the power of a pre-trained text-to-video generator within an inversion-then-generation framework. To realize motion editing while preserving source video content, based on the insights that temporal and spatial self-attention layers encode inter-frame and intra-frame dependency respectively, we introduce auxiliary motion-reference and reconstruction branches to produce text-guided motion and source features respectively. The obtained features are then injected into the main editing path via temporal and spatial self-attention layers. Extensive experiments demonstrate that UniEdit covers video motion editing and various appearance editing scenarios, and surpasses the state-of-the-art methods. Our code will be publicly available.

Figures (18)

• Figure 1: Examples edited by UniEdit. Our solution supports both video motion editing in the time axis (i.e., from playing guitar to eating or waving) and various video appearance editing scenarios (i.e., stylization, rigid/non-rigid object replacement, background modification). We encourage the readers to watch the videos on our https://jianhongbai.github.io/UniEdit/.
• Figure 2: Overview of UniEdit. It follows an inversion-then-generation pipeline and consists of a main editing path, an auxiliary reconstruction branch and an auxiliary motion-reference branch. The reconstruction branch produces source features for content preservation, and the motion-reference branch yields text-guided motion features for motion injection. The source features and motion features are injected into the main editing path through spatial self-attention (SA-S) and temporal self-attention (SA-T) modules respectively (Sec. \ref{['sec_motion_editing']}). We further introduce spatial structure control to retain the coarse structure of the source video (Sec. \ref{['sec_appearance_editing']}).
• Figure 3: Detailed illustration of the relationship between the main editing path, the auxiliary reconstruction branch and the auxiliary motion-reference branch. The content preservation, motion injection and spatial structure control are achieved by the fusion of $Q$ (query), $K$ (key), $V$ (value) features in spatial self-attention (SA-S) and temporal self-attention (SA-T) modules.
• Figure 4: Examples edited by UniEdit. For each case, the upper frames come from the source video, and the lower frames indicate the edited results with the target prompt. We encourage the readers to watch the https://jianhongbai.github.io/UniEdit/ and make evaluations.
• Figure 5: Comparison with state-of-the-art methods for both video motion and appearance editing. It shows that UniEdit achieves better source content preservation, and outperforms baselines in motion editing by a large margin.