Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Approximate Message-Passing for Compressed Sensing with Sublinear Sparsity

Keigo Takeuchi

TL;DR

This work extends generalized approximate message-passing (GAMP) to signals with sublinear sparsity, establishing a state-evolution framework that tracks unnormalized squared error in the sublinear regime under novel inner-denoiser assumptions. It develops Bayesian GAMP by leveraging spike-and-slab priors for the inner denoiser and a Bayes-optimal outer denoiser, with extreme-value theory used to justify the inner-denoising steps. Theoretical SE results characterize a reconstruction threshold $\delta^{*}$ governing asymptotically exact recovery, and numerical experiments show Bayesian GAMP outperforming existing sublinear-sparsity methods for linear and 1-bit compressed sensing. The findings highlight the practical potential of sublinear-sparsity recovery and provide a rigorous link between Bayesian denoising, SE, and sample complexity in generalized measurements.

Abstract

This paper addresses the reconstruction of an unknown signal vector with sublinear sparsity from generalized linear measurements. Generalized approximate message-passing (GAMP) is proposed via state evolution in the sublinear sparsity limit, where the signal dimension $N$, measurement dimension $M$, and signal sparsity $k$ satisfy $\log k/\log N\to γ\in[0, 1)$ and $M/\{k\log (N/k)\}\toδ$ as $N$ and $k$ tend to infinity. While the overall flow in state evolution is the same as that for linear sparsity, each proof step for inner denoising requires stronger assumptions than those for linear sparsity. The required new assumptions are proved for Bayesian inner denoising. When Bayesian outer and inner denoisers are used in GAMP, the obtained state evolution recursion is utilized to evaluate the prefactor $δ$ in the sample complexity, called reconstruction threshold. If and only if $δ$ is larger than the reconstruction threshold, Bayesian GAMP can achieve asymptotically exact signal reconstruction. In particular, the reconstruction threshold is finite for noisy linear measurements when the support of non-zero signal elements does not include a neighborhood of zero. As numerical examples, this paper considers linear measurements and 1-bit compressed sensing. Numerical simulations for both cases show that Bayesian GAMP outperforms existing algorithms for sublinear sparsity in terms of the sample complexity.

Generalized Approximate Message-Passing for Compressed Sensing with Sublinear Sparsity

TL;DR

This work extends generalized approximate message-passing (GAMP) to signals with sublinear sparsity, establishing a state-evolution framework that tracks unnormalized squared error in the sublinear regime under novel inner-denoiser assumptions. It develops Bayesian GAMP by leveraging spike-and-slab priors for the inner denoiser and a Bayes-optimal outer denoiser, with extreme-value theory used to justify the inner-denoising steps. Theoretical SE results characterize a reconstruction threshold governing asymptotically exact recovery, and numerical experiments show Bayesian GAMP outperforming existing sublinear-sparsity methods for linear and 1-bit compressed sensing. The findings highlight the practical potential of sublinear-sparsity recovery and provide a rigorous link between Bayesian denoising, SE, and sample complexity in generalized measurements.

Abstract

This paper addresses the reconstruction of an unknown signal vector with sublinear sparsity from generalized linear measurements. Generalized approximate message-passing (GAMP) is proposed via state evolution in the sublinear sparsity limit, where the signal dimension , measurement dimension , and signal sparsity satisfy and as and tend to infinity. While the overall flow in state evolution is the same as that for linear sparsity, each proof step for inner denoising requires stronger assumptions than those for linear sparsity. The required new assumptions are proved for Bayesian inner denoising. When Bayesian outer and inner denoisers are used in GAMP, the obtained state evolution recursion is utilized to evaluate the prefactor in the sample complexity, called reconstruction threshold. If and only if is larger than the reconstruction threshold, Bayesian GAMP can achieve asymptotically exact signal reconstruction. In particular, the reconstruction threshold is finite for noisy linear measurements when the support of non-zero signal elements does not include a neighborhood of zero. As numerical examples, this paper considers linear measurements and 1-bit compressed sensing. Numerical simulations for both cases show that Bayesian GAMP outperforms existing algorithms for sublinear sparsity in terms of the sample complexity.
Paper Structure (71 sections, 13 theorems, 307 equations, 6 figures)

This paper contains 71 sections, 13 theorems, 307 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Compressed Sensing
  3. Linear Sparsity
  4. Linear Measurement
  5. Nonlinear Measurements
  6. Sublinear Sparsity
  7. Linear Measurement
  8. Nonlinear Measurements
  9. Contributions
  10. Organization
  11. Notation
  12. Generalized Approximate Message-Passing
  13. Main Results
  14. Assumptions and Definitions
  15. State Evolution
  16. ...and 56 more sections

Key Result

Theorem 1

Define $\bar{\xi}_{\mathrm{in},t}$ in (z_t) and (v_in) as (xi_in_bar) and suppose that Assumptions assumption_x--assumption_inner hold. Then, the unnormalized square error $\|\hat{\boldsymbol{x}}_{t+1} - \boldsymbol{x}\|_{2}^{2}$ for GAMP converges in probability to $\bar{v}_{\mathrm{in},t+1}$---giv

Figures (6)

  • Figure 1: The expected unnormalized square error $\mathbb{E}[\|\boldsymbol{X} - f_{X}(\boldsymbol{Y}; v_{\tilde{N}})\|_{2}^{2}]$ versus $\log_{2}N$ in Lemma \ref{['lemma_MSE']} for $k=N^{0.25}$. The horizontal dashed lines show the asymptotic expression (\ref{['unnormalized_square_error']}) in the sublinear sparsity limit.
  • Figure 2: EXIT-like chart of Bayesian AMP for the linear measurement and $1/\sigma^{2}=40$ dB. The outer module shows the straight line (\ref{['linear_v_out_bar']}) while the inner module represents (\ref{['Bayes_v_in_bar']}) for $M/\{k\log (N/k)\}\to\delta$ in the sublinear sparsity limit. The chart is independent of the parameter $\log k/\log N\to\gamma\in[0, 1)$.
  • Figure 3: Unnormalized square error versus $\delta=M/\{k\log (N/k)\}$ for the linear measurement, $k=16$, $N=2^{16}$, and $1/\sigma^{2}=40$ dB. Bayesian AMP with $20$ iterations is compared to FISTA with backtracking Beck09, gradient-based restart O'Donoghue15, $10^{3}$ iterations, and optimized $\lambda$ in (\ref{['Lasso']}) for each $\delta$, as well as OMP Tropp07 with $k$ iterations. The vertical dotted line shows the weak reconstruction threshold in Definition \ref{['def_weak_reconstruction_threshold']} for Bayesian AMP. $10^{4}$ independent trials were simulated for all algorithms, with the only exception of $10^{5}$ trials for Bayesian AMP with $\delta\approx 1.33$.
  • Figure 4: Unnormalized square error versus the number of iterations for the linear measurement, $k=16$, $N=2^{16}$, and $1/\sigma^{2}=40$ dB. Bayesian AMP is compared to OMP Tropp07 and FISTA with backtracking Beck09, gradient-based restart O'Donoghue15, and optimized $\lambda=\sqrt{0.8\sigma^{2}M^{-1}\log N}$Wainwright091. $10^{5}$ independent trials were simulated for Bayesian AMP while $10^{4}$ independent trials were for OMP and FISTA.
  • Figure 5: EXIT-like chart of Bayesian GAMP for 1-bit compressed sensing and $\sigma^{2}=0$. The outer module shows the inverse function of (\ref{['1bCS_v_out_bar']}) while the inner module represents (\ref{['Bayes_v_in_bar']}) for $M/\{k\log (N/k)\}\to\delta$ in the sublinear sparsity limit. The chart is independent of the parameter $\log k/\log N\to\gamma\in[0, 1)$.
  • ...and 1 more figures

Theorems & Definitions (18)

  • Remark 1
  • Definition 1: Pseudo-Lipschitz Function
  • Definition 2: Empirical Convergence
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Corollary 1
  • ...and 8 more