FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution

TL;DR

FlashVSR introduces a diffusion-based one-step streaming framework for real-time video super-resolution, addressing latency, scalability, and high-resolution generalization. It fuses a three-stage distillation pipeline, locality-constrained sparse attention, and a tiny conditional decoder to achieve 17 FPS at 768×1408 on a single A100 and up to 12× speedups over prior one-step diffusion VSR methods. The approach is underpinned by VSR-120K, a large-scale joint image–video dataset, enabling robust training and evaluation. Empirical results demonstrate superior quality and efficiency, with strong performance at ultra-high resolutions and practical streaming latency, and code/dataset will be released to promote reproducibility and further research.

Abstract

Diffusion models have recently advanced video restoration, but applying them to real-world video super-resolution (VSR) remains challenging due to high latency, prohibitive computation, and poor generalization to ultra-high resolutions. Our goal in this work is to make diffusion-based VSR practical by achieving efficiency, scalability, and real-time performance. To this end, we propose FlashVSR, the first diffusion-based one-step streaming framework towards real-time VSR. FlashVSR runs at approximately 17 FPS for 768x1408 videos on a single A100 GPU by combining three complementary innovations: (i) a train-friendly three-stage distillation pipeline that enables streaming super-resolution, (ii) locality-constrained sparse attention that cuts redundant computation while bridging the train-test resolution gap, and (iii) a tiny conditional decoder that accelerates reconstruction without sacrificing quality. To support large-scale training, we also construct VSR-120K, a new dataset with 120k videos and 180k images. Extensive experiments show that FlashVSR scales reliably to ultra-high resolutions and achieves state-of-the-art performance with up to 12x speedup over prior one-step diffusion VSR models. We will release the code, pretrained models, and dataset to foster future research in efficient diffusion-based VSR.

Figures (12)

• Figure 1: Efficiency and performance comparison in high-resolution video restoration. Compared to state-of-the-art VSR models (e.g., DOVE and SeedVR2-3B), FlashVSR restores sharper textures and more detailed structures. It achieves near real-time performance at 17 FPS on $768 \times 1408$ videos using a single A100 GPU, corresponding to a measured 11.8$\times$ speedup over the fastest one-step diffusion VSR model. (Zoom in for best view)
• Figure 2: Overview of the three-stage training pipeline of FlashVSR, covering video–image joint SR training, adaptation with block-sparse causal attention for streaming inference, and distribution-matching one-step distillation combined with reconstruction supervision.
• Figure 3: Locality-Constrained Sparse Attention. Left: At ultra-high resolutions, performing inference beyond the trained positional encoding range produces artifacts (e.g., repetition or blur). Restricting each query to a local attention window keeps the positional encoding range consistent between training and inference, thereby preventing artifacts. Right: Two local window rules, namely boundary-preserved and boundary-truncated, are illustrated. The final sparse attention mask is computed within these local masks.
• Figure 4: Training pipeline of the TC Decoder.
• Figure 5: Visualization results of video super-resolution on real-world and AIGC videos.