Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonasymptotic Convergence Rates for Plug-and-Play Methods With MMSE Denoisers

Henry Pritchard, Rahul Parhi

Abstract

It is known that the minimum-mean-squared-error (MMSE) denoiser under Gaussian noise can be written as a proximal operator, which suffices for asymptotic convergence of plug-and-play (PnP) methods but does not reveal the structure of the induced regularizer or give convergence rates. We show that the MMSE denoiser corresponds to a regularizer that can be written explicitly as an upper Moreau envelope of the negative log-marginal density, which in turn implies that the regularizer is 1-weakly convex. Using this property, we derive (to the best of our knowledge) the first sublinear convergence guarantee for PnP proximal gradient descent with an MMSE denoiser. We validate the theory with a one-dimensional synthetic study that recovers the implicit regularizer. We also validate the theory with imaging experiments (deblurring and computed tomography), which exhibit the predicted sublinear behavior.

Nonasymptotic Convergence Rates for Plug-and-Play Methods With MMSE Denoisers

Abstract

It is known that the minimum-mean-squared-error (MMSE) denoiser under Gaussian noise can be written as a proximal operator, which suffices for asymptotic convergence of plug-and-play (PnP) methods but does not reveal the structure of the induced regularizer or give convergence rates. We show that the MMSE denoiser corresponds to a regularizer that can be written explicitly as an upper Moreau envelope of the negative log-marginal density, which in turn implies that the regularizer is 1-weakly convex. Using this property, we derive (to the best of our knowledge) the first sublinear convergence guarantee for PnP proximal gradient descent with an MMSE denoiser. We validate the theory with a one-dimensional synthetic study that recovers the implicit regularizer. We also validate the theory with imaging experiments (deblurring and computed tomography), which exhibit the predicted sublinear behavior.

Paper Structure

This paper contains 15 sections, 8 theorems, 46 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proximal Gradient Descent
  3. Plug-and-Play Methods
  4. Related Works
  5. Contributions
  6. Preliminaries
  7. Main Results
  8. New Characterization of the MMSE Regularizer
  9. Proximal Gradient Descent Convergence Results
  10. Experiments
  11. Numerical Illustration of \ref{['thm:mmse_prior']}
  12. DNN Architecture for Inverse Problems
  13. Gaussian Deblurring
  14. Computed Tomography
  15. Conclusion

Key Result

Lemma 1

Let $f:\mathbb{R}^n \to (-\infty,\infty]$. Then, $M^\gamma M_\gamma f(x) \leq f(x)$ for all $x\in \mathbb{R}^n$ and $M^\gamma f$ is $\tfrac{1}{\gamma}$-weakly convex.

Figures (6)

  • Figure 1: Top: Calculated implicit regularizer in the MMSE denoiser ($\phi_{\mathrm{MMSE}}$), negative log-marginal $f_Z$, and negative log-prior $f_X$ for three mixture-of-Gaussian priors under unit Gaussian noise. Bottom: The learned regularizer $\phi^{\mathrm{learned}}_{\mathrm{MMSE}}$, along with the same reference curves. Details in \ref{['section:illustration']}.
  • Figure 2: Top: Calculated implicit regularizer in the MMSE denoiser ($\phi_{\mathrm{MMSE}}$), negative log-marginal $f_Z$, and negative log-prior $f_X$ for three mixture-of-Laplacian priors under unit Gaussian noise. Bottom: The learned regularizer $\phi^{\mathrm{learned}}_{\mathrm{MMSE}}$, along with the same reference curves. Details in \ref{['section:illustration']}.
  • Figure 3: Best-iterate gradient norm and iterate difference over $100$ iterations of PnP-PGD with an MMSE denoiser for the MNIST Gaussian deblurring experiment in \ref{['section:deblurring']}, plotted on a log-log scale. Both quantities decay sublinearly, consistent with a $\mathcal{O}(1/\sqrt{K})$ rate.
  • Figure 4: PnP-PGD reconstruction results on three selections from the MNIST dataset under Gaussian blur (\ref{['section:deblurring']}). From left to right: clean reference, blurred and noisy input, and reconstruction after 100 iterations. PSNR gains are roughly $13$--$14$ dB are achieved across all three examples.
  • Figure 5: Best-iterate gradient norm and iterate difference over $1000.0$ iterations of PnP-PGD with an MMSE denoiser for the MayoCT Computed Tomography experiment in \ref{['section:CT']}, plotted on a log-log scale. Both quantities decay sublinearly, consistent with a $\mathcal{O}(1/\sqrt{K})$ rate.
  • ...and 1 more figures

Theorems & Definitions (20)

  • Definition 1: Lower Moreau Envelope
  • Definition 2: Upper Moreau Envelope
  • Definition 3: Weak convexity
  • Lemma 1
  • proof
  • Definition 4: Proximal operator
  • Theorem 2.1: Extension of the Moreau Gradient Identity
  • proof
  • Theorem 2.2: Tweedie's Formula
  • Theorem 2.3: Main result from gribonval_should and gribonval_reconciling
  • ...and 10 more