Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SURF: Signature-Retained Fast Video Generation

Kaixin Ding, Xi Chen, Sihui Ji, Yuan Gao, Liang Hou, Xin Tao, Hengshuang Zhao

Abstract

The demand for high-resolution video generation is growing rapidly. However, the generation resolution is severely constrained by slow inference speeds. For instance, Wan2.1 requires over 50 minutes to generate a single 720p video. While previous works explore accelerating video generation from various aspects, most of them compromise the distinctive signatures (e.g., layout, semantic, motion) of the original model. In this work, we propose SURF, an efficient framework for generating high-resolution videos, while maximally keeping the signatures. Specifically, SURF divides video generation into two stages: First, we leverage the pretrained model to infer at optimal resolution and downsample latent to generate low-resolution previews in fast speed; then we design a Refiner to upscale the preview. In the preview stage, we identify that directly inferring a model (trained with higher resolution) on lower resolution causes severe losses in signatures. So we introduce noise reshifting, a training-free technique that mitigates this issue by conducting initial denoising steps on the original resolution and switching to low resolution in later steps. In the refine stage, we establish a mapping relationship between the preview and the high-resolution target, which significantly reduces the denoising steps. We further integrate shifting windows and carefully design the training paradigm to get a powerful and efficient Refiner. In this way, SURF enables generating high-resolution videos efficiently while maximally closer to the signatures of the given pretrained model. SURF is conceptually simple and could serve as a plug-in that is compatible with various base model and acceleration methods. For example, it achieves 12.5x speedup for generating 5-second, 16fps, 720p Wan 2.1 videos and 8.7x speedup for generating 5-second, 24fps, 720p HunyuanVideo.

SURF: Signature-Retained Fast Video Generation

Abstract

The demand for high-resolution video generation is growing rapidly. However, the generation resolution is severely constrained by slow inference speeds. For instance, Wan2.1 requires over 50 minutes to generate a single 720p video. While previous works explore accelerating video generation from various aspects, most of them compromise the distinctive signatures (e.g., layout, semantic, motion) of the original model. In this work, we propose SURF, an efficient framework for generating high-resolution videos, while maximally keeping the signatures. Specifically, SURF divides video generation into two stages: First, we leverage the pretrained model to infer at optimal resolution and downsample latent to generate low-resolution previews in fast speed; then we design a Refiner to upscale the preview. In the preview stage, we identify that directly inferring a model (trained with higher resolution) on lower resolution causes severe losses in signatures. So we introduce noise reshifting, a training-free technique that mitigates this issue by conducting initial denoising steps on the original resolution and switching to low resolution in later steps. In the refine stage, we establish a mapping relationship between the preview and the high-resolution target, which significantly reduces the denoising steps. We further integrate shifting windows and carefully design the training paradigm to get a powerful and efficient Refiner. In this way, SURF enables generating high-resolution videos efficiently while maximally closer to the signatures of the given pretrained model. SURF is conceptually simple and could serve as a plug-in that is compatible with various base model and acceleration methods. For example, it achieves 12.5x speedup for generating 5-second, 16fps, 720p Wan 2.1 videos and 8.7x speedup for generating 5-second, 24fps, 720p HunyuanVideo.
Paper Structure (12 sections, 3 equations, 7 figures, 5 tables)

This paper contains 12 sections, 3 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Overall Pipeline.
  5. Preview stage
  6. Refine stage
  7. Experiments
  8. Implementation Details
  9. Comprehensive Evaluations
  10. Stage-wise Visualization
  11. Ablation
  12. Conclusion

Figures (7)

  • Figure 1: Demonstration for signature retention. Distilled model loses the signatures of the base model, which could lead to misaligned body parts and weak semantic consistency. Our method maintains the layout, semantics, and motion of base model with huge speed up.
  • Figure 2: Overview of the SURF. Our framework employs a powerful pretrained model (e.g., Wan 2.1) and a lightweight Refiner network. Together, they execute the denoising process through an OptimRes-LowRes-HighRes flow (SURF), producing outputs with signature (e.g., layout, semantic, motion) closely match those of the base model (Wan 2.1), while achieving huge speed up.
  • Figure 3: Shift window across adjacent blocks.
  • Figure 4: User study. We report pair-wise preference rates. SURF achieves comparable quality to Wan 2.1 with huge speed up.
  • Figure 5: Qualitative comparisons. Our method achieves up to 12× speedup while maintaining signatures of base model. Unreasonable contents are marked in orange. Rather than aimless camera panning, SURF generates high fidelity videos with semantically aligned motion.
  • ...and 2 more figures