Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SUNLayer: Stable denoising with generative networks

Ruhui Jin, Dustin G. Mixon, Soledad Villar

TL;DR

This work proposes a theoretical setting that uses spherical harmonics to identify what mathematical properties of the activation functions will allow signal denoising with local methods, and focuses on the classical signal processing problem of image Denoising.

Abstract

Deep neural networks are often used to implement powerful generative models for real-world data. Notable applications include image denoising, as well as other classical inverse problems like compressed sensing and super-resolution. To provide a rigorous but simplified analysis of generative models, in this work, we introduce an elegant theoretical framework based on spherical harmonics, namely \textbf{SUNLayer}. Our theoretical framework identifies explicit conditions on activation functions that guarantee denoising under local optimization. Numerical experiments examine the theoretical properties on commonly used activation functions and demonstrate their stable denoising performance.

SUNLayer: Stable denoising with generative networks

TL;DR

This work proposes a theoretical setting that uses spherical harmonics to identify what mathematical properties of the activation functions will allow signal denoising with local methods, and focuses on the classical signal processing problem of image Denoising.

Abstract

Deep neural networks are often used to implement powerful generative models for real-world data. Notable applications include image denoising, as well as other classical inverse problems like compressed sensing and super-resolution. To provide a rigorous but simplified analysis of generative models, in this work, we introduce an elegant theoretical framework based on spherical harmonics, namely \textbf{SUNLayer}. Our theoretical framework identifies explicit conditions on activation functions that guarantee denoising under local optimization. Numerical experiments examine the theoretical properties on commonly used activation functions and demonstrate their stable denoising performance.

Paper Structure

This paper contains 22 sections, 9 theorems, 63 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Main contributions
  4. SUNLayer: a neural network model
  5. Denoising
  6. Preliminaries: spherical harmonics
  7. Analysis
  8. Noiseless case
  9. Denoising
  10. Numerical experiments
  11. Denoising a generative model for MNIST
  12. Gegenbauer approximations of common activation functions
  13. Denoising in a synthetic framework
  14. Denoising for tomography-type data
  15. Generative network architecture.
  16. ...and 7 more sections

Key Result

Proposition 1

$\mathcal{H}_k(S^{n})$ is a finite dimensional space and

Figures (9)

  • Figure 1: Denoising with generative priors
  • Figure 2: Denoising performance of the SUNLayer for different activation functions
  • Figure 3: Activation functions and their Gegenbauer approximations for $K=30$ and $n=2$.
  • Figure 4: Activation functions and their Gegenbauer approximations for $K=30$ and $n=10$.
  • Figure 5: The dashed arrow indicates that stability tests are performed on the decoder outputs as a function of latent perturbations.
  • ...and 4 more figures

Theorems & Definitions (23)

  • Definition 1: Spherical harmonics
  • Proposition 1
  • Definition 2
  • Proposition 2
  • Lemma 1
  • proof
  • Definition 3
  • Lemma 2
  • proof
  • Lemma 3
  • ...and 13 more