Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasons for the Superiority of Stochastic Estimators over Deterministic Ones: Robustness, Consistency and Perceptual Quality

Guy Ohayon, Theo Adrai, Michael Elad, Tomer Michaeli

TL;DR

The paper tackles ill-posed image restoration by contrasting deterministic and stochastic estimators through formal notions of perceptual quality and consistency. It proves that only a posterior sampler can achieve perfect perceptual quality under perfect consistency, implying deterministic consistent estimators cannot reach this ideal. Empirically, it shows that enforcing robustness on deterministic mappings degrades perceptual quality, while stochastic methods can be robust and maintain high perceptual fidelity, with robustness also enhancing output diversity. Extensive experiments on inpainting and super-resolution with GAN-based restorers (including SRFlow) demonstrate that stochastic models achieve high perceptual quality with strong robustness, whereas deterministic models suffer from a robustness–perceptual quality trade-off. The work advocates prioritizing stochastic restoration approaches for better, more reliable recovery in ill-posed imaging problems.

Abstract

Stochastic restoration algorithms allow to explore the space of solutions that correspond to the degraded input. In this paper we reveal additional fundamental advantages of stochastic methods over deterministic ones, which further motivate their use. First, we prove that any restoration algorithm that attains perfect perceptual quality and whose outputs are consistent with the input must be a posterior sampler, and is thus required to be stochastic. Second, we illustrate that while deterministic restoration algorithms may attain high perceptual quality, this can be achieved only by filling up the space of all possible source images using an extremely sensitive mapping, which makes them highly vulnerable to adversarial attacks. Indeed, we show that enforcing deterministic models to be robust to such attacks profoundly hinders their perceptual quality, while robustifying stochastic models hardly influences their perceptual quality, and improves their output variability. These findings provide a motivation to foster progress in stochastic restoration methods, paving the way to better recovery algorithms.

Reasons for the Superiority of Stochastic Estimators over Deterministic Ones: Robustness, Consistency and Perceptual Quality

TL;DR

The paper tackles ill-posed image restoration by contrasting deterministic and stochastic estimators through formal notions of perceptual quality and consistency. It proves that only a posterior sampler can achieve perfect perceptual quality under perfect consistency, implying deterministic consistent estimators cannot reach this ideal. Empirically, it shows that enforcing robustness on deterministic mappings degrades perceptual quality, while stochastic methods can be robust and maintain high perceptual fidelity, with robustness also enhancing output diversity. Extensive experiments on inpainting and super-resolution with GAN-based restorers (including SRFlow) demonstrate that stochastic models achieve high perceptual quality with strong robustness, whereas deterministic models suffer from a robustness–perceptual quality trade-off. The work advocates prioritizing stochastic restoration approaches for better, more reliable recovery in ill-posed imaging problems.

Abstract

Stochastic restoration algorithms allow to explore the space of solutions that correspond to the degraded input. In this paper we reveal additional fundamental advantages of stochastic methods over deterministic ones, which further motivate their use. First, we prove that any restoration algorithm that attains perfect perceptual quality and whose outputs are consistent with the input must be a posterior sampler, and is thus required to be stochastic. Second, we illustrate that while deterministic restoration algorithms may attain high perceptual quality, this can be achieved only by filling up the space of all possible source images using an extremely sensitive mapping, which makes them highly vulnerable to adversarial attacks. Indeed, we show that enforcing deterministic models to be robust to such attacks profoundly hinders their perceptual quality, while robustifying stochastic models hardly influences their perceptual quality, and improves their output variability. These findings provide a motivation to foster progress in stochastic restoration methods, paving the way to better recovery algorithms.
Paper Structure (32 sections, 1 theorem, 25 equations, 8 figures, 4 tables)

This paper contains 32 sections, 1 theorem, 25 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Problem setting and preliminaries
  3. Perceptual quality
  4. Consistency
  5. Can deterministic estimators be consistent and achieve high perceptual quality?
  6. Erratic behavior of deterministic estimators
  7. Perceptual quality approximation
  8. Deterministic GAN estimators
  9. Robustness of stochastic estimators
  10. Experiments on natural images
  11. Demonstrations with GANs
  12. Experimental setup
  13. Analysis of the results
  14. Robustness of SRFlow
  15. Summary
  16. ...and 17 more sections

Key Result

Theorem 3.1

For a deterministic degradation, $y=D(x)$, an estimator ${\hat{X}}$ is perfectly consistent ($p_{Y|X}=p_{Y|\hat{X}}$) and achieves perfect perceptual quality ($p_{\hat{X}}=p_{X}$) if and only if it is the posterior sampler $p_{\hat{X}|Y}=p_{X|Y}$.

Figures (8)

  • Figure 1: Output samples from several consistent restoration algorithms that solve the image inpainting task on the CelebA data set. The erratic stochastic and deterministic algorithms are trained solely with a GAN loss. As can be seen, the erratic stochastic estimator barely produces output variability per input, which reveals a tendency of mode collapse of CGANs ganpix2pix2017mathieu2016deepdsganICLR2019pscgan. The robust algorithms are trained to also defend against adversarial attacks by adding \ref{['eq:stochastic-robustness-practical']} to the GAN objective. Robustifying the deterministic algorithm deteriorates its perceptual quality, while doing so for the stochastic algorithm preserves this quality and significantly improves its output variability. Refer to \ref{['tab:inpainting-sto-vs-det']} for quantitative evaluation.
  • Figure 2: The main conclusions of this paper, summarized in a flow chart. We only consider perfectly consistent restoration algorithms. A deterministic restoration algorithm has, by definition, no output variability. Such an algorithm attains low perceptual quality if it is robust, and may attain high perceptual quality otherwise. A stochastic restoration algorithm may always attain high perceptual quality, whether it is robust or not. In the case where the stochastic algorithm is a neural network trained as a GAN, robustifying such an algorithm improves (increases) its output variation per input. Without robustification, such an algorithm often attains low output variability, and essentially behaves as a non-robust (erratic), deterministic algorithm.
  • Figure 3: An illustration on a toy restoration problem of the tradeoff between robustness and perceptual quality for deterministic restoration algorithms, as well as as a demonstration that such a tradeoff does not exist for stochastic algorithms. Each column corresponds to a different estimator, with the estimator's name indicated in the title of the column. Top row: the support of the data distribution (gray disc), and the support of the output distribution of each estimator (a continuous colored line of 10,000 random samples from the output distribution of each estimator). Bottom row: 100 random samples from the data distribution (gray dots), and 100 random samples from the output distribution of each estimator (colored stars). The precision and recall improved_precision_recall of each estimator are computed between 1000 random output samples from the output distribution of the estimator and 1000 random samples from the data distribution. Refer to \ref{['sec:space-filling']} for more details.
  • Figure 4: Quantitative evaluation of the super resolution algorithms described in \ref{['sec:experimental-setup']}. AI-PSNR is equal to 34dB for all algorithms. The anticipated link between robustness and perceptual quality is revealed again: The robust deterministic models achieve the lowest perceptual quality for all scaling factors. As the scaling factor (degradation severity) increases, we observe higher output variability only for the robust stochastic models, and a larger perceptual quality gap for the robust deterministic models.
  • Figure 5: Mental illustration of a tradeoff between precision and recall for consistent, high perceptual quality, continuous deterministic estimators. In the top row we show the support of the data distribution and of the distribution of each estimator's outputs. In the bottom row we show 100 random samples from these distributions. $\hat{X}_{1}$ avoids the forbidden region (the middle empty ellipse which is not part of $\text{Supp}(p_{X})$), and therefore attains almost perfect precision with impaired recall. $\hat{X}_{2}$ passes through the forbidden region in order to attain higher recall, but as a result compromises on precision since it generates outputs that are not in the data distribution.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Theorem 3.1
  • proof