Denoising guarantees for optimized sampling schemes in compressed sensing

TL;DR

This work analyzes compressed sensing where signals lie in a union of low-dimensional subspaces, including sparse and generative priors, under Gaussian measurement noise. It proves a denoising phenomenon for optimized sampling schemes, showing that the reconstruction error decreases as $m^{-1/2}$ when the number of measurements grows, and that the sampling distribution can be tuned via local coherence to minimize sample complexity. The authors introduce the optimized sampling vector $p'$ with $p'_i = \alpha_i^2/\|\boldsymbol{\alpha}\|_2^2$, establish RIP-based guarantees for unitary-structured measurements, and extend results to with-replacement sampling and arbitrary priors. Numerically, the denoising behavior matches the theory for both generative priors (via neural nets) and sparse priors, demonstrating practical impact for designing robust, efficient sensing systems.

Abstract

Compressed sensing with subsampled unitary matrices benefits from \emph{optimized} sampling schemes, which feature improved theoretical guarantees and empirical performance relative to uniform subsampling. We provide, in a first of its kind in compressed sensing, theoretical guarantees showing that the error caused by the measurement noise vanishes with an increasing number of measurements for optimized sampling schemes, assuming that the noise is Gaussian. We moreover provide similar guarantees for measurements sampled with-replacement with arbitrary probability weights. All our results hold on prior sets contained in a union of low-dimensional subspaces. Finally, we demonstrate that this denoising behavior appears in empirical experiments with a rate that closely matches our theoretical guarantees when the prior set is the range of a generative ReLU neural network and when it is the set of sparse vectors.

Figures (4)

• Figure 1: a) Relative error for optimized sampling versus uniform sampling with generative models. b) the red channel of the local coherence.
• Figure 2: Relative recovery error for 256 repeated experiments at each $m$ and $\sigma$. We perform least-squares fits in log space for $m \in [10^3.9, 10^5.4]$ (roughly after the phase transition, but before saturation). The slopes of the fitted lines from top to bottom are $-0.56,-0.50,-0.37,-0.29$ respectively.
• Figure 3: Noisy faces recovered by optimized sampling and uniform sampling.
• Figure 4: Relative errors of five signal recovery experiments for each value of $m$ and $\sigma$. We perform least-squares fits in log space for $m>300$ (roughly after the phase transition). The fits have slopes of $-1.11$, $-0.67$, $-0.53$, $-0.53$, $-0.54$ respectively, in order of decreasing $\sigma$. The ambient dimension is $n=6.6 \cdot 10^4$, we run experiments up to $m \approx 4n$.

Theorems & Definitions (39)

• Definition 2.1: Local coherence
• Remark 2.2
• Definition 2.3: Sampling matrix
• Definition 2.4: With-replacement sampling matrix
• Definition 2.6: Optimized sampling vector
• Theorem 2.7: Compressed sensing with optimized sampling
• Remark 2.8
• Remark 2.9
• Definition 3.1: Unit truncation
• Definition 3.2: Restricted Isometry Property