GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration

TL;DR

GenDeg introduces a diffusion-model–based framework to synthesize diverse real-world degradations on clean images, addressing poor OoD generalization in All-In-One Image Restoration. It conditions a latent diffusion model on the clean image, a text prompt, and degradation intensity to generate degraded variants, yielding approximately $550{,}000$ synthetic samples, which are combined with existing data into GenDS (~$750{,}000$ pairs) across six degradations. Empirical results show significant OoD improvements for multiple AIOR models trained on GenDS, bridging the gap between training and unseen real-world scenarios. This work demonstrates the value of diffusion-generated degradations for robust restoration and offers controllable, large-scale data augmentation for AIOR pipelines.

Abstract

Deep learning-based models for All-In-One Image Restoration (AIOR) have achieved significant advancements in recent years. However, their practical applicability is limited by poor generalization to samples outside the training distribution. This limitation arises primarily from insufficient diversity in degradation variations and scenes within existing datasets, resulting in inadequate representations of real-world scenarios. Additionally, capturing large-scale real-world paired data for degradations such as haze, low-light, and raindrops is often cumbersome and sometimes infeasible. In this paper, we leverage the generative capabilities of latent diffusion models to synthesize high-quality degraded images from their clean counterparts. Specifically, we introduce GenDeg, a degradation and intensity-aware conditional diffusion model capable of producing diverse degradation patterns on clean images. Using GenDeg, we synthesize over 550k samples across six degradation types: haze, rain, snow, motion blur, low-light, and raindrops. These generated samples are integrated with existing datasets to form the GenDS dataset, comprising over 750k samples. Our experiments reveal that image restoration models trained on the GenDS dataset exhibit significant improvements in out-of-distribution performance compared to those trained solely on existing datasets. Furthermore, we provide comprehensive analyses on implications of diffusion model-based synthetic degradations for AIOR.

Figures (7)

• Figure 1: Out-of-distribution performance of three image restoration models when trained solely using existing datasets and our proposed GenDS dataset. Significant improvements can be observed across all degradations. Metric values reduce outward.
• Figure 2: Analysis of real and synthetic image restoration datasets for various degradations. Existing datasets are small and less diverse, especially for haze, low-light, and raindrop. Our diffusion-generated synthetic data substantially increases the number of samples as well as scene diversity.
• Figure 3: t-SNE visualization of degradation features obtained for hazy samples from existing training data, GenDS dataset and OoD test sets. The features were obtained using DA-CLIP daclip.
• Figure 4: (a) Illustrates the training stage of the GenDeg model where it is trained to condition on the clean image, text prompt and mean intensity ($\mu$) and variation ($\sigma$) of the degradation pattern. (b) Shows the inference stage where the model generates a degraded image based on these conditions; and (c) Depicts the architecture of the Swin-transformer-based restoration network.
• Figure 5: Qualitative comparisons of image restoration models trained with and without our GenDS dataset. The suffix GD represents training with the GenDS dataset. Zoomed-in patches are provided for viewing fine details.