Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms

Mengyu Zhao, Xi Chen, Xin Yuan, Shirin Jalali

TL;DR

This work analyzes untrained neural networks for snapshot compressive imaging (SCI) and introduces SCI-BDVP, a bagged deep-video prior approach that combines multi-scale, untrained priors within a PGD-style descent to robustly recover high-dimensional SCI data from a single 2D measurement. Theoretical results connect DIP-based SCI performance to problem dimensions and mask properties, yielding bounds on the number of frames recoverable and guidance for mask design, with extensions to noisy measurements. Empirically, SCI-BDVP achieves state-of-the-art performance among UNN methods in noiseless video SCI and outperforms both supervised and UNN baselines under noise, demonstrating strong robustness and practical effectiveness. The paper also provides a detailed analysis of mask optimization, showing that optimal mask sparsity shifts with noise and dataset characteristics, and offers a comprehensive framework for future SCI systems leveraging untrained priors.

Abstract

Snapshot compressive imaging (SCI) recovers high-dimensional (3D) data cubes from a single 2D measurement, enabling diverse applications like video and hyperspectral imaging to go beyond standard techniques in terms of acquisition speed and efficiency. In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure. Such UNN-based methods are appealing as they have the potential of avoiding the computationally intensive retraining required for different source models and different measurement scenarios. We first develop a theoretical framework for characterizing the performance of such UNN-based methods. The theoretical framework, on the one hand, enables us to optimize the parameters of data-modulating masks, and on the other hand, provides a fundamental connection between the number of data frames that can be recovered from a single measurement to the parameters of the untrained NN. We also employ the recently proposed bagged-deep-image-prior (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by standard UNN solutions. Our experimental results show that in video SCI our proposed solution achieves state-of-the-art among UNN methods, and in the case of noisy measurements, it even outperforms supervised solutions.

Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms

TL;DR

This work analyzes untrained neural networks for snapshot compressive imaging (SCI) and introduces SCI-BDVP, a bagged deep-video prior approach that combines multi-scale, untrained priors within a PGD-style descent to robustly recover high-dimensional SCI data from a single 2D measurement. Theoretical results connect DIP-based SCI performance to problem dimensions and mask properties, yielding bounds on the number of frames recoverable and guidance for mask design, with extensions to noisy measurements. Empirically, SCI-BDVP achieves state-of-the-art performance among UNN methods in noiseless video SCI and outperforms both supervised and UNN baselines under noise, demonstrating strong robustness and practical effectiveness. The paper also provides a detailed analysis of mask optimization, showing that optimal mask sparsity shifts with noise and dataset characteristics, and offers a comprehensive framework for future SCI systems leveraging untrained priors.

Abstract

Snapshot compressive imaging (SCI) recovers high-dimensional (3D) data cubes from a single 2D measurement, enabling diverse applications like video and hyperspectral imaging to go beyond standard techniques in terms of acquisition speed and efficiency. In this paper, we focus on SCI recovery algorithms that employ untrained neural networks (UNNs), such as deep image prior (DIP), to model source structure. Such UNN-based methods are appealing as they have the potential of avoiding the computationally intensive retraining required for different source models and different measurement scenarios. We first develop a theoretical framework for characterizing the performance of such UNN-based methods. The theoretical framework, on the one hand, enables us to optimize the parameters of data-modulating masks, and on the other hand, provides a fundamental connection between the number of data frames that can be recovered from a single measurement to the parameters of the untrained NN. We also employ the recently proposed bagged-deep-image-prior (bagged-DIP) idea to develop SCI Bagged Deep Video Prior (SCI-BDVP) algorithms that address the common challenges faced by standard UNN solutions. Our experimental results show that in video SCI our proposed solution achieves state-of-the-art among UNN methods, and in the case of noisy measurements, it even outperforms supervised solutions.
Paper Structure (31 sections, 7 theorems, 83 equations, 13 figures, 4 tables, 1 algorithm)

This paper contains 31 sections, 7 theorems, 83 equations, 13 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions of this Work
  3. Theoretical:
  4. Algorithmic:
  5. Notations
  6. Related Work
  7. Mask optimization.
  8. DIP for SCI inverse problem
  9. SCI inverse problem
  10. Theoretical analysis of DIP-based SCI recovery
  11. Noise-free measurements
  12. Noisy measurements
  13. SCI-BDVP: Bagged-DVP for video SCI
  14. SCI-BDVP
  15. SCI-BDVP: Descent step
  16. ...and 16 more sections

Key Result

Theorem 3.1

Let ${\bf x}\in\mathcal{Q}$. Assume that $g_{\theta}({\bf u}):[0,1]^p\to\mathbb{R}^{nB}$ is $L$-Lipschitz as a function of $\theta$. Let ${\bf y}={\bf H}{\bf x}$, where ${\bf H}=[{\bf D}_1,\ldots,{\bf D}_B]$, where ${\bf D}_i=\mathop{\rm diag}\nolimits(D_{i,1},\ldots,D_{i,n})$, $i=1,\ldots,B$, are i with a probability larger than $1- 2^{-0.5 k\log\log n+1}$.

Figures (13)

  • Figure 1: PSNR, shown as y-axis, of $\|{\bf x}-{\bf \hat{x}}\|_2$, $\|{\bf \hat{x}}-\bar{{\bf x}}_B\|_2$ and $\|{\bf x}-\bar{{\bf x}}_B\|_2$: masks are generated as $\mathrm{Bern}(p)$, $p$ shown as x-axis,. Blue, orange and green lines represent noise levels of $\sigma=0$, $10$ and $25$, respectively. Solid black line shows $\|{\bf x}-\bar{{\bf x}}_B\|_2$. Solid colored lines and dashed colored lines represent $\|{\bf x}-{\bf \hat{x}}\|_2$ and $\|{\bf \hat{x}}-\bar{{\bf x}}_B\|_2$, respectively.
  • Figure 2: SCI-BDVP (GD): Iterative PGD-type algorithm. Each step consists of GD and BDVP projection, with an additional skip-connection.
  • Figure 3: SCI-BDVP consisting of $K$ individual DVPs trained separately.
  • Figure 4: Network structure of DVP we use in SCI-BDVP.
  • Figure 5: Reconstruction PSNR ($\|{\bf x}-{\bf \hat{x}}\|_2$) and SSIM as a function of $p$, using SCI-BDVP (GAP) (two leftmost figures) and PnP-FastDVDnet (GAP) (two rightmost figures). For each value of $p$, the masks are independently generated i.i.d.$\sim \mathrm{Bern}(p)$.
  • ...and 8 more figures

Theorems & Definitions (9)

  • Theorem 3.1
  • Corollary 3.2
  • Corollary 3.3
  • Theorem 3.4
  • Theorem 3.5
  • Corollary 3.6
  • Remark 4.1
  • Lemma 6.1: Concentration of $\chi^2$ JalaliM:14-MEP-IT
  • Definition 6.1