Proxies for Distortion and Consistency with Applications for Real-World Image Restoration

TL;DR

Proxies for Distortion and Consistency with Applications for Real-World Image Restoration tackles restoring images with unknown forward models by introducing a degradation estimator that predicts the degradation chain $A$ from degraded input measurements. This estimator enables an empirical log-likelihood approximation, a blind consistency metric, and a plug-and-play diffusion-based restoration method (ELAD) guided by likelihood, along with no-reference distortion proxies ProxMSE and ProxLPIPS to rank restorations without ground-truth. The approach is validated on synthetic and real-world blind-face restoration datasets, showing improved consistency and competitive distortion and perceptual quality, while enabling dataset synthesis that mirrors real degradations. Overall, the framework provides a practical, end-to-end toolkit for designing, evaluating, and comparing blind real-world image restoration methods in the absence of paired data.

Abstract

Real-world image restoration deals with the recovery of images suffering from an unknown degradation. This task is typically addressed while being given only degraded images, without their corresponding ground-truth versions. In this hard setting, designing and evaluating restoration algorithms becomes highly challenging. This paper offers a suite of tools that can serve both the design and assessment of real-world image restoration algorithms. Our work starts by proposing a trained model that predicts the chain of degradations a given real-world measured input has gone through. We show how this estimator can be used to approximate the consistency -- the match between the measurements and any proposed recovered image. We also use this estimator as a guiding force for the design of a simple and highly-effective plug-and-play real-world image restoration algorithm, leveraging a pre-trained diffusion-based image prior. Furthermore, this work proposes no-reference proxy measures of MSE and LPIPS, which, without access to the ground-truth images, allow ranking of real-world image restoration algorithms according to their (approximate) MSE and LPIPS. The proposed suite provides a versatile, first of its kind framework for evaluating and comparing blind image restoration algorithms in real-world scenarios.

Figures (14)

• Figure 1: This work introduces several novel tools to help tackle the challenging task of real-world image restoration. We propose an estimator that predicts the degradations a real-world corrupted measurement has gone through. Using this estimator, we approximate the consistency of any reconstructed candidate with a given input measurement, and use such a measure to develop a plug-and-play real-world restoration algorithm. Moreover, we propose blind (no-reference) measures of distortion that mimic MSE and LPIPS, thus eliminating the need for ground-truth images when comparing the distortion of real-world image restoration algorithms.
• Figure 2: Degradation estimator accuracy. We test our degradation estimator on synthetic CelebA-Test datasets (\ref{['sec:deg_est']}). (a-d) Scatter plots and $R^2$ scores of the true vs. the predicted values for each type of operator in \ref{['eq:bfr_model']}. (e) The mean and standard deviation of the PSNR between $\mu_{Y}({x}, {a})$ and $\mu_{Y}({x}, {a}_{\theta}({y}))$. The estimator demonstrates high prediction accuracy, as reflected by the high PSNR and $R^2$ scores.
• Figure 3: Degradations in real-world BFR datasets. Using our degradation estimator, we reveal the distribution of the degradations presented in real-world datasets. This information can be utilized to better analyze such datasets, as well as mimicking them.
• Figure 4: Proxy consistency measure. Each plot shows the CMSE versus ProxCMSE, evaluated on synthetic CelebA-Test datasets (\ref{['sec:deg_est']}). The strong alignment of the two suggests that ProxCMSE is a trustworthy approximation for the CMSE when the degradation process is unknown.
• Figure 5: Consistency of P&P real-world restoration methods. Left column: A synthetic degraded example from CelebA-Test. First row: The ground-truth image alongside the restorations of ELAD (our method), DifFace, and PGDiff. Second row: The mean of the likelihood defined using the true degradation process ${a}$ and the corresponding image from the row above. For reference, we also report the MSE and CMSE of each method (\ref{['eq:cmse']}). As shown, ELAD attains better consistency with the measurement. PGDiff, for example, does not restore the ear ( rectangle), which is visible both in the degraded image and in $\mu_{Y}({x},a)$. Similarly, DifFace erases part of the brow ( rectangle) and creates inconsistent artifacts in the hat ( rectangle). In contrast, ELAD performs better by guiding the diffusion process to produce consistent reconstructions. As a positive side effect, achieving better consistency also leads to better distortion, which is apparent visually in the top row, and can be confirmed by the reported MSE and by \ref{['tab:elad']}.

Theorems & Definitions (5)

• Proposition 1
• Proposition 1
• proof
• Proposition 2
• proof