Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features

TL;DR

This paper tackles the limitations of diffusion-model–based video super-resolution (VSR), notably error accumulation, artifacts, and the fidelity-perceptual trade-off caused by imperfect frame alignment. It introduces DGAF-VSR, whichDensely guides aligned features through an Optical Guided Warping Module (OGWM) and a Feature-wise Temporal Condition Module (FTCM), grounded in two key observations: (i) feature-domain guidance yields stronger spatial–temporal correlations than pixel-domain guidance, and (ii) warping at an upscaled resolution better preserves high-frequency information with an optimal rescaling factor. Through extensive experiments on REDS4, Vid4, and VideoLQ, DGAF-VSR achieves state-of-the-art results across perceptual quality, fidelity, and temporal consistency, outperforming both non-DM and DM-based baselines (e.g., substantial gains in DISTS, PSNR, and tLPIPS). The proposed approach provides a practical, efficient pathway to high-fidelity, temporally coherent VSR and suggests broader applicability to other low-level video restoration tasks with straightforward diffusion-model adaptations.

Abstract

Diffusion model (DM) based Video Super-Resolution (VSR) approaches achieve impressive perceptual quality. However, they suffer from error accumulation, spatial artifacts, and a trade-off between perceptual quality and fidelity, primarily caused by inaccurate alignment and insufficient compensation between video frames. In this paper, within the DM-based VSR pipeline, we revisit the role of alignment and compensation between adjacent video frames and reveal two crucial observations: (a) the feature domain is better suited than the pixel domain for information compensation due to its stronger spatial and temporal correlations, and (b) warping at an upscaled resolution better preserves high-frequency information, but this benefit is not necessarily monotonic. Therefore, we propose a novel Densely Guided diffusion model with Aligned Features for Video Super-Resolution (DGAF-VSR), with an Optical Guided Warping Module (OGWM) to maintain high-frequency details in the aligned features and a Feature-wise Temporal Condition Module (FTCM) to deliver dense guidance in the feature domain. Extensive experiments on synthetic and real-world datasets demonstrate that DGAF-VSR surpasses state-of-the-art methods in key aspects of VSR, including perceptual quality (35.82\% DISTS reduction), fidelity (0.20 dB PSNR gain), and temporal consistency (30.37\% tLPIPS reduction).

Figures (12)

• Figure 1: Illustration of average SSIM, PSNR, $F(H)$ and $F(\sigma)$ metrics for adjacent variables across all frames in the REDS4 dataset, shown for diffusion steps (a) $t=\textit{1}$ and (b) $t=\textit{981}$.
• Figure 1: Illustration of edge strength across features aligned through different strategies, using different edge detection operators including the Canny, Sobel, and Laplacian, from top to bottom. (a) the upscaling and downscaling operations, (b) the warping operation on features of different resolutions, and (c) the rescaling-based warping strategy.
• Figure 2: Illustration of the edge strength and the high-pass strength under different alignment strategies. The impact of (a) the upscaling and downscaling operations, (b) the warping operation on features of different resolutions, and (c) the upscaling–warping–downscaling strategy.
• Figure 2: Illustration of high-pass strength and edge strength across different diffusion steps.
• Figure 3: The framework of the proposed DGAF-VSR. The network is composed of a Flow prediction Module (FM), $T$ diffusion steps, and a VAE decoder. The FM module estimates bidirectional optical flow estimation. Each diffusion step consists of one Optical Guided Warping Module (OGWM) module for feature warping process, and one Feature-wise Temporal Condition Module (FTCM) module for fine-grained information integration through dense feature guidance.