Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Learning ISTA

Fabio Valerio Massoli, Christos Louizos, Arash Behboodi

TL;DR

The paper tackles sparse recovery under underdetermined forward models with unknown dictionaries and sample-specific sensing matrices by introducing A-DLISTA, an adaptive LISTA variant, and VLISTA, a variational framework that places a distribution over dictionaries. A-DLISTA uses a shared augmentation network to adapt per-layer thresholds and step sizes to the current measurement setup, improving recovery under non-static Φ; VLISTA embeds A-DLISTA as the likelihood and jointly learns dictionary distributions via a hierarchical probabilistic model, enabling uncertainty quantification and OOD detection. The authors provide theoretical motivation and extensive experiments on synthetic data and image datasets (MNIST, CIFAR-10), showing that A-DLISTA generally outperforms non-adaptive baselines and that VLISTA offers calibrated uncertainties with competitive performance. The work advances fast, adaptable sparse recovery in scenarios with varying forward operators and contributes a principled Bayesian perspective to dictionary learning within unfolded networks, with practical implications for adaptive sensing, imaging, and communication systems.

Abstract

Compressed sensing combines the power of convex optimization techniques with a sparsity-inducing prior on the signal space to solve an underdetermined system of equations. For many problems, the sparsifying dictionary is not directly given, nor its existence can be assumed. Besides, the sensing matrix can change across different scenarios. Addressing these issues requires solving a sparse representation learning problem, namely dictionary learning, taking into account the epistemic uncertainty of the learned dictionaries and, finally, jointly learning sparse representations and reconstructions under varying sensing matrix conditions. We address both concerns by proposing a variant of the LISTA architecture. First, we introduce Augmented Dictionary Learning ISTA (A-DLISTA), which incorporates an augmentation module to adapt parameters to the current measurement setup. Then, we propose to learn a distribution over dictionaries via a variational approach, dubbed Variational Learning ISTA (VLISTA). VLISTA exploits A-DLISTA as the likelihood model and approximates a posterior distribution over the dictionaries as part of an unfolded LISTA-based recovery algorithm. As a result, VLISTA provides a probabilistic way to jointly learn the dictionary distribution and the reconstruction algorithm with varying sensing matrices. We provide theoretical and experimental support for our architecture and show that our model learns calibrated uncertainties.

Variational Learning ISTA

TL;DR

The paper tackles sparse recovery under underdetermined forward models with unknown dictionaries and sample-specific sensing matrices by introducing A-DLISTA, an adaptive LISTA variant, and VLISTA, a variational framework that places a distribution over dictionaries. A-DLISTA uses a shared augmentation network to adapt per-layer thresholds and step sizes to the current measurement setup, improving recovery under non-static Φ; VLISTA embeds A-DLISTA as the likelihood and jointly learns dictionary distributions via a hierarchical probabilistic model, enabling uncertainty quantification and OOD detection. The authors provide theoretical motivation and extensive experiments on synthetic data and image datasets (MNIST, CIFAR-10), showing that A-DLISTA generally outperforms non-adaptive baselines and that VLISTA offers calibrated uncertainties with competitive performance. The work advances fast, adaptable sparse recovery in scenarios with varying forward operators and contributes a principled Bayesian perspective to dictionary learning within unfolded networks, with practical implications for adaptive sensing, imaging, and communication systems.

Abstract

Compressed sensing combines the power of convex optimization techniques with a sparsity-inducing prior on the signal space to solve an underdetermined system of equations. For many problems, the sparsifying dictionary is not directly given, nor its existence can be assumed. Besides, the sensing matrix can change across different scenarios. Addressing these issues requires solving a sparse representation learning problem, namely dictionary learning, taking into account the epistemic uncertainty of the learned dictionaries and, finally, jointly learning sparse representations and reconstructions under varying sensing matrix conditions. We address both concerns by proposing a variant of the LISTA architecture. First, we introduce Augmented Dictionary Learning ISTA (A-DLISTA), which incorporates an augmentation module to adapt parameters to the current measurement setup. Then, we propose to learn a distribution over dictionaries via a variational approach, dubbed Variational Learning ISTA (VLISTA). VLISTA exploits A-DLISTA as the likelihood model and approximates a posterior distribution over the dictionaries as part of an unfolded LISTA-based recovery algorithm. As a result, VLISTA provides a probabilistic way to jointly learn the dictionary distribution and the reconstruction algorithm with varying sensing matrices. We provide theoretical and experimental support for our architecture and show that our model learns calibrated uncertainties.
Paper Structure (39 sections, 5 theorems, 35 equations, 14 figures, 7 tables, 2 algorithms)

This paper contains 39 sections, 5 theorems, 35 equations, 14 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Related Works
  4. Background
  5. Sparse linear inverse problems
  6. LISTA
  7. Method
  8. Augmented Dictionary Learning ISTA (A-DLISTA)
  9. Variational Learning ISTA
  10. Prior Model
  11. Posterior Model
  12. Likelihood Model
  13. Training Objective
  14. Experiments
  15. Datasets and Baselines
  16. ...and 24 more sections

Key Result

Proposition 4.1

Suppose that ${\bm{y}}^k={\bm{\Phi}}^k{\bm{\Psi}}_o{\bm{x}}_*$, where ${\bm{x}}_*$ is the ground truth sparse vector with support $\text{supp}({\bm{x}}_*)=S$, and ${\bm{\Psi}}_o$ is the ground truth dictionary. For DLISTA iterations given as we have:

Figures (14)

  • Figure 1: Models architectures. Left:A-DLISTA architecture. Each blue block represents a single ISTA-like iteration parametrized by the dictionary ${\bm{\Psi}}_t$, the threshold and step size $\{\theta_t^i, \gamma_t^i\}$. The red blocks represent the augmentation network (with shared parameters across layers) that adapts $\{\theta_t^i, \gamma_t^i\}$ for layer $t$ based on the dictionary ${\bm{\Psi}}_t$ and the current measurement setup ${\bm{\Phi}}^i$ for the $i-$th data sample. Right:VLISTA (inference) architecture. The red and blue blocks correspond to the same operations as for A-DLISTA. The pink blocks represent the posterior model used to refine the dictionary based on input data $\{{\bm{y}}^i, {\bm{\Phi}}^i\}$ and the sparse vector reconstructed at layer $t$, $\mathbf{x}_t$.
  • Figure 2: VLISTA graphical model. Dependencies on ${\bm{y}}^i$ and ${\bm{\Phi}}^i$ are factored out for simplicity. The sampling is done only based on the posterior $q_{\bm{\phi}}({\bm{\Psi}}_t|{\bm{x}}_{t-1},{\bm{y}}^i,{\bm{\Phi}}^i)$. Dashed lines represent variational approximations.
  • Figure 3: NMSE's median. The $y$-axes is in dB (the lower the better) for a different number of measurements ($x$-axes).
  • Figure 4: p-value for OOD rejection as a function of the noise level. The green line represents a reference p-value equal to 0.05.
  • Figure 5: Augmentation model's architecture for A-DLISTA.
  • ...and 9 more figures

Theorems & Definitions (8)

  • Proposition 4.1
  • Theorem B.1
  • Remark B.2
  • Proposition B.3
  • Lemma B.4
  • proof
  • Lemma B.5
  • proof