RBPGAN: Recurrent Back-Projection GAN for Video Super Resolution

TL;DR

This work tackles the challenge of achieving temporally coherent yet spatially detailed video super-resolution (VSR). It introduces RBPGAN, a GAN that combines the RBPN generator with a TecoGAN-inspired spatio-temporal discriminator and a Ping-Pong loss to enforce long-term temporal consistency. The model leverages RBPN's back-projection-based refinement for high-frequency details while leveraging a temporal discriminator to stabilize frame-to-frame coherence, all within a compact architecture. Evaluations on Vimeo-derived training data and standard benchmarks (Vid4, ToS3) demonstrate improved temporal cohesion (lower LPIPS) with competitive PSNR/SSIM, highlighting the practical potential for more stable, high-quality VSR under hardware constraints. The approach advances temporally aware VSR by fusing proven spatial-detail methods with temporal coherence strategies in a self-supervised training setup.

Abstract

Recently, video super resolution (VSR) has become a very impactful task in the area of Computer Vision due to its various applications. In this paper, we propose Recurrent Back-Projection Generative Adversarial Network (RBPGAN) for VSR in an attempt to generate temporally coherent solutions while preserving spatial details. RBPGAN integrates two state-of-the-art models to get the best in both worlds without compromising the accuracy of produced video. The generator of the model is inspired by RBPN system, while the discriminator is inspired by TecoGAN. We also utilize Ping-Pong loss to increase temporal consistency over time. Our contribution together results in a model that outperforms earlier work in terms of temporally consistent details, as we will demonstrate qualitatively and quantitatively using different datasets.

Figures (5)

• Figure 1: RBPN Architecture
• Figure 2: Discrimenator Architectue
• Figure 3: RBPGAN Architectue
• Figure 4: walk (Top:LR , Bottom: RBPGAN)
• Figure 5: calendar (Top:LR , Bottom: RBPGAN)