Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse Bayesian Generative Modeling for Compressive Sensing

Benedikt Böck, Sadaf Syed, Wolfgang Utschick

TL;DR

This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior that is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem.

Abstract

This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS and, in particular, sparse Bayesian learning (SBL), to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability from generative models to training data. However, unlike most state-of-the-art generative models, it is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem. Additionally, similar to Dirichlet prior networks, our model parameterizes a conjugate prior enabling its application for uncertainty quantification. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.

Sparse Bayesian Generative Modeling for Compressive Sensing

TL;DR

This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior that is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem.

Abstract

This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS and, in particular, sparse Bayesian learning (SBL), to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability from generative models to training data. However, unlike most state-of-the-art generative models, it is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem. Additionally, similar to Dirichlet prior networks, our model parameterizes a conjugate prior enabling its application for uncertainty quantification. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.

Paper Structure

This paper contains 56 sections, 2 theorems, 45 equations, 11 figures, 3 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Background and Method
  3. Notation.
  4. Problem Formulation
  5. Sparse Bayesian Learning for Compressive Sensing
  6. Gaussian Mixture Models and Variational Autoencoders
  7. Proposed Method
  8. Training principle.
  9. Estimation scheme.
  10. Theoretical Analysis and Specific Parameterizations
  11. Sparsity Guarantees
  12. Compressive Sensing
  13. Compressive Sensing
  14. Discussion
  15. Limitation.
  16. ...and 41 more sections

Key Result

Theorem 3.1

Let $p_{\bm{\theta}}(\bm{s}|\bm{z}) = \mathcal{N}(\bm{s};\bm{0},\text{diag}(\bm{\gamma}_{\bm{\theta}}(\bm{z}))$ (i.e., it is defined according to eq:intr_stat_model), let $\bm{z}$ be either continuous or discrete and finite, and let $\bm{\gamma}_{\bm{\theta}}(\bm{z}) > \bm{0}$. Then, there exists a The integral corresponds to a summation for discrete and finite $p_{\bm{\delta}}(\bm{z})$.

Figures (11)

  • Figure 1: A schematic of the sparsity inducing CSVAE.
  • Figure 2: a) and b) $\mathrm{nMSE}$ and $\mathrm{SSIM}$ over $M$ ($N_t = 20000$, MNIST), c) and d) $\mathrm{nMSE}$ and $\mathrm{SSIM}$ over $N_t$ ($M = 160$, MNIST), e) exemplary reconstructed MNIST images ($M=200$, $N_t=20000$), f) $\mathrm{nMSE}$ over $M$ ($\mathrm{SNR}_{\text{dB}} = 10\mathrm{dB}$, $N_t = 10000$, piece-wise smooth fct.), g) $\mathrm{nMSE}$ over $N_t$ ($\mathrm{SNR}_{\text{dB}} = 10\mathrm{dB}$, $M = 100$, piece-wise smooth fct.), h) exemplary reconstructed piece-wise smooth fct. ($M=100$, $N_t=1000$), i) $\mathrm{nMSE}$ comparison of dictionaries (MNIST, $M=160$, $N_t=20000$).
  • Figure 3: a) and b) $\mathrm{nMSE}$ and $\mathrm{SSIM}$ over $M$ ($N_t = 5000$), c) and d) $\mathrm{nMSE}$ and $\mathrm{SSIM}$ over $N_t$ ($M = 1800$), e) exemplary reconstructed celebA images ($M=2700$, $N_t=5000$), f) histogram of $h(\bm{z}|\bm{y})$ for compressed test MNIST images of digits $0$,$1$ and $7$, where the CSVAE is trained on compressed zeros, g) training and reconstruction time for MNIST ($M = 200$, $N_t = 20000$).
  • Figure 4: Exemplary signals within the 1D dataset of piece-wise smooth functions.
  • Figure 5: Wavelet Transforms of the exemplary signals in Fig. \ref{['fig:1D_examples']}.
  • ...and 6 more figures

Theorems & Definitions (2)

  • Theorem 3.1
  • Lemma 3.2