DreamStyle: A Unified Framework for Video Stylization

TL;DR

DreamStyle tackles the fragmented landscape of video stylization by unifying text-guided, style-image-guided, and first-frame-guided inputs within a single V2V/I2V framework. It leverages a data curation pipeline to produce high-quality paired video data and employs a token-specific LoRA design to disentangle multiple conditioning signals while preserving base-model capabilities. The two-stage training on large- and high-quality datasets yields strong style consistency, structure preservation, and dynamic video quality across tasks, with demonstrated extensions to multi-style fusion and long-video stylization. This approach advances practical, flexible stylization for diverse user needs and lays groundwork for scalable multimodal video editing. $L( heta)$ is optimized across three conditioned tasks, enabling unified inference without sacrificing performance in any single modality.

Abstract

Video stylization, an important downstream task of video generation models, has not yet been thoroughly explored. Its input style conditions typically include text, style image, and stylized first frame. Each condition has a characteristic advantage: text is more flexible, style image provides a more accurate visual anchor, and stylized first frame makes long-video stylization feasible. However, existing methods are largely confined to a single type of style condition, which limits their scope of application. Additionally, their lack of high-quality datasets leads to style inconsistency and temporal flicker. To address these limitations, we introduce DreamStyle, a unified framework for video stylization, supporting (1) text-guided, (2) style-image-guided, and (3) first-frame-guided video stylization, accompanied by a well-designed data curation pipeline to acquire high-quality paired video data. DreamStyle is built on a vanilla Image-to-Video (I2V) model and trained using a Low-Rank Adaptation (LoRA) with token-specific up matrices that reduces the confusion among different condition tokens. Both qualitative and quantitative evaluations demonstrate that DreamStyle is competent in all three video stylization tasks, and outperforms the competitors in style consistency and video quality.

Figures (9)

• Figure 1: We propose DreamStyle, a unified video stylization framework, which provides a flexible and practical tool for users to create high-quality stylized videos. Given an input video and the reference styles in forms of text, style image, or stylized first frame, DreamStyle faithfully generates videos that align with the desired style—while preserving the main content of the input video.
• Figure 2: Data Curation Pipeline. We propose generating the training data with two key steps: image stylization followed by image to video. Considering the characteristics of different image stylization techniques, we construct a CT dataset and a SFT dataset, where SDXL (equipped with ControlNet, InstantStyle, and ID plugin) and Seedream 4.0 are selected as their stylization models, respectively. For image to video, we utilize ControlNets to enhance the motion consistency between the generated stylized and raw videos. To ensure the data quality, we additionally apply automatic filtering for CT data and manual filtering for SFT data.
• Figure 3: Example that depth fails to capture accurate detail. (a) The raw video frame, (b) the extracted depth map, (c) the generated realistic frame, (d) the generated stylized frame.
• Figure 4: Overview of DreamStyle Framework. DreamStyle is built on the Wan14B-I2V model, integrating the text and raw-video conditions through the cross-attention and image channels of the base model, while the first-frame and style-image conditions serve as additional frames concatenated to the start and end of the frame sequence. We train it using a standard flow matching loss and a token-specific LoRA that contributes to distinguishing different condition tokens.
• Figure 5: Qualitative comparison on three video stylization tasks.