Towards Realistic Data Generation for Real-World Super-Resolution

TL;DR

RealDGen tackles the Real SR generalization problem by decoupling content and degradation and leveraging a content-degradation diffusion model conditioned on unpaired HR and real LR data. It pre-trains robust content and degradation extractors, then trains a Decoupled DDPM with a modulation block, followed by a data-generation pipeline that yields realistic LR samples aligned with real-world degradations. Across RealSR, DRealSR, and smartphone benchmarks, RealDGen delivers consistent gains for both PSNR/SSIM–oriented and perceptual SR models, and a user study confirms improved perceived realism. The approach enables scalable, unsupervised generation of realistic paired data, facilitating better generalization to diverse real-world degradations without extensive manual data collection.

Abstract

Existing image super-resolution (SR) techniques often fail to generalize effectively in complex real-world settings due to the significant divergence between training data and practical scenarios. To address this challenge, previous efforts have either manually simulated intricate physical-based degradations or utilized learning-based techniques, yet these approaches remain inadequate for producing large-scale, realistic, and diverse data simultaneously. In this paper, we introduce a novel Realistic Decoupled Data Generator (RealDGen), an unsupervised learning data generation framework designed for real-world super-resolution. We meticulously develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model to create realistic low-resolution images from unpaired real LR and HR images. Extensive experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations, significantly advancing the performance of popular SR models on various real-world benchmarks.

Figures (14)

• Figure 1: (a) and (b) are SR performance on different train data and degradation distribution of different methods. (c) is the pipeline of our unsupervised data generation framework RealDGen.
• Figure 2: An overview of the training pipeline of our proposed RealDGen. We first train on the content and degradation extractors, then train Decoupled DDPM while fine-tuning the partial parameters of the extractors. RealDGen adaptively generates realistic LR images with arbitrarily given real LR images and unpaired HR images.
• Figure 3: Visual comparison of generated LR. Our method achieves the best visual results with realistic degradation and high fidelity. Please zoom in for better visualization.
• Figure 4: Visual comparison of Real SR based on different data generation methods. Real SR training using our data achieves the best visual results. Please zoom in for better visualization.
• Figure 5: Visual comparison of generated low-resolution images with different reference real LR images. Please zoom in for better visualization.