DDR: Exploiting Deep Degradation Response as Flexible Image Descriptor

TL;DR

Deep Degradation Response is presented, a method to quantify changes in image deep features under varying degradation conditions that facilitates flexible and adaptive degradation, enabling the controlled synthesis of image degradation through text-driven prompts.

Abstract

Image deep features extracted by pre-trained networks are known to contain rich and informative representations. In this paper, we present Deep Degradation Response (DDR), a method to quantify changes in image deep features under varying degradation conditions. Specifically, our approach facilitates flexible and adaptive degradation, enabling the controlled synthesis of image degradation through text-driven prompts. Extensive evaluations demonstrate the versatility of DDR as an image descriptor, with strong correlations observed with key image attributes such as complexity, colorfulness, sharpness, and overall quality. Moreover, we demonstrate the efficacy of DDR across a spectrum of applications. It excels as a blind image quality assessment metric, outperforming existing methodologies across multiple datasets. Additionally, DDR serves as an effective unsupervised learning objective in image restoration tasks, yielding notable advancements in image deblurring and single-image super-resolution. Our code is available at: https://github.com/eezkni/DDR

Figures (11)

• Figure 1: Example of Degradation Response Variations. We apply the same level of Gaussian Blur to different images from the LIVEitw LIVEitw dataset. $x$ and $x_d$ denote the original and degraded images, respectively. $l(\cdot, \cdot)$ is the LPIPS metric zhang2018unreasonable between $x$ and $x_d$, which measures the extent of changes in the feature space. The results demonstrate that images with different content and texture characteristics exhibit varying degrees of change.
• Figure 1: Comparison of SRCC for blur degradation on the LIVE LIVE dataset.$\text{DDR}_{blur}$ refers to the deep feature response to blur obtained by manually synthesizing degradation in the pixel domain. $\text{DDR}_{blur}$ demonstrates highest correlation with human opinion.
• Figure 2: Distribution of DDR on the LIVEitw LIVEitw dataset. "Low", "optimal", and "high" refer to different levels of handcrafted degradation applied in the pixel domain, while "adaptive" represents adaptively fusing text-driven degradation in the feature domain. The DDR with "optimal" and "adaptive" degradation achieve significantly better performance on the BIQA task.
• Figure 3: The framework of our proposed DDR with two different degradation fusing methods. (a) Synthesizing degradation with a handcrafted process in the pixel domain. (b) Fusing text-driven degradation in the feature domain.
• Figure 4: Images with high and low DDR to different degradation types. We measure DDR with five types of degradation by setting their corresponding prompt pair. We observe that image with lower DDR to a specific type of degradation is likely to obtain this degradation of a higher level.