Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Orthogonal Approximate Message-Passing for Sublinear Sparsity

Keigo Takeuchi

TL;DR

The paper develops Generalized Orthogonal AMP (GOAMP) for reconstructing sublinearly sparse signals from generalized linear measurements by introducing an Onsager-corrected, two-module architecture whose estimation errors become asymptotically Gaussian under a unified state-evolution framework. It provides Bayesian GOAMP with Bayes-optimal outer denoisers, establishes state evolution results, and proves a reconstruction threshold for linear measurements matching Bayes-AMP for standard Gaussian matrices. Numerical experiments on linear and 1-bit compressed sensing show that Bayesian GOAMP outperforms existing algorithms, particularly for ill-conditioned sensing matrices. The framework offers a rigorous pathway to analyze long-memory message-passing in sublinear sparsity and demonstrates practical reconstruction gains in challenging sensing scenarios.

Abstract

This paper addresses the reconstruction of sparse signals from generalized linear measurements. Signal sparsity is assumed to be sublinear in the signal dimension while it was proportional to the signal dimension in conventional research. Approximate message-passing (AMP) has poor convergence properties for sensing matrices beyond standard Gaussian matrices. To solve this convergence issue, generalized orthogonal AMP (GOAMP) is proposed for signals with sublinear sparsity. The main feature of GOAMP is the so-called Onsager correction to realize asymptotic Gaussianity of estimation errors. The Onsager correction in GOAMP is designed via state evolution for orthogonally invariant sensing matrices in the sublinear sparsity limit, where the signal sparsity and measurement dimension tend to infinity at sublinear speed in the signal dimension. When the support of non-zero signals does not contain a neighborhood of the origin, GOAMP using Bayesian denoisers is proved to achieve error-free signal reconstruction for linear measurements if and only if the measurement dimension is larger than a threshold, which is equal to that of AMP for standard Gaussian sensing matrices. Numerical simulations are also presented for linear measurements and 1-bit compressed sensing. When ill-conditioned sensing matrices are used, GOAMP for sublinear sparsity is shown to outperform existing reconstruction algorithms, including generalized AMP for sublinear sparsity.

Generalized Orthogonal Approximate Message-Passing for Sublinear Sparsity

TL;DR

The paper develops Generalized Orthogonal AMP (GOAMP) for reconstructing sublinearly sparse signals from generalized linear measurements by introducing an Onsager-corrected, two-module architecture whose estimation errors become asymptotically Gaussian under a unified state-evolution framework. It provides Bayesian GOAMP with Bayes-optimal outer denoisers, establishes state evolution results, and proves a reconstruction threshold for linear measurements matching Bayes-AMP for standard Gaussian matrices. Numerical experiments on linear and 1-bit compressed sensing show that Bayesian GOAMP outperforms existing algorithms, particularly for ill-conditioned sensing matrices. The framework offers a rigorous pathway to analyze long-memory message-passing in sublinear sparsity and demonstrates practical reconstruction gains in challenging sensing scenarios.

Abstract

This paper addresses the reconstruction of sparse signals from generalized linear measurements. Signal sparsity is assumed to be sublinear in the signal dimension while it was proportional to the signal dimension in conventional research. Approximate message-passing (AMP) has poor convergence properties for sensing matrices beyond standard Gaussian matrices. To solve this convergence issue, generalized orthogonal AMP (GOAMP) is proposed for signals with sublinear sparsity. The main feature of GOAMP is the so-called Onsager correction to realize asymptotic Gaussianity of estimation errors. The Onsager correction in GOAMP is designed via state evolution for orthogonally invariant sensing matrices in the sublinear sparsity limit, where the signal sparsity and measurement dimension tend to infinity at sublinear speed in the signal dimension. When the support of non-zero signals does not contain a neighborhood of the origin, GOAMP using Bayesian denoisers is proved to achieve error-free signal reconstruction for linear measurements if and only if the measurement dimension is larger than a threshold, which is equal to that of AMP for standard Gaussian sensing matrices. Numerical simulations are also presented for linear measurements and 1-bit compressed sensing. When ill-conditioned sensing matrices are used, GOAMP for sublinear sparsity is shown to outperform existing reconstruction algorithms, including generalized AMP for sublinear sparsity.

Paper Structure

This paper contains 48 sections, 16 theorems, 300 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Sparse Signal Reconstruction
  3. Lasso-Type Algorithms
  4. Message-Passing Algorithms
  5. Contributions
  6. Organization
  7. Notation
  8. Generalized OAMP
  9. Overview
  10. Modules
  11. Initialization
  12. Denoisers in module A
  13. Orthogonal Approximate Message-Passing
  14. Main Results
  15. State Evolution
  16. ...and 33 more sections

Key Result

Theorem 1

Suppose that Assumptions assumption_w--assumption_SE are satisfied. Assume that $v_{\mathrm{B}}^{\mathrm{x},t+1}$ in the inner sub-module for module B is a consistent estimator of $\|\boldsymbol{x}_{\mathrm{B}}^{t+1} - \boldsymbol{x}\|_{2}^{2}$ in the sublinear sparsity limit. Furthermore, replace $

Figures (4)

  • Figure 1:
  • Figure 2: State evolution chart of Bayesian OAMP for the linear measurement, $\delta=1.5$, and $P/\sigma^{2}=40$ dB. Module A shows (\ref{['moduleA_x_var_bar_linear']}) while module B represents (\ref{['moduleB_x_var_bar_Bayes']}). The zigzag line shows the trajectory of Bayesian OAMP for $\kappa=5000$. The state evolution recursion does not depend on $\gamma$ in $k=N^{\gamma}$.
  • Figure 3: Squared norm for the normalized errors versus the condition number $\kappa$ for 1-bit compressed sensing, $N=2^{16}$, $k=16$, $M=2000$, and $\sigma^{2}=0$. The damping factor in Bayesian GOAMP was set to $(\theta_{\mathrm{x}}, \theta_{\mathrm{z}})=(0.6, 0.6)$ for $\kappa>1$ while $(\theta_{\mathrm{x}}, \theta_{\mathrm{z}})=(0.7, 1)$ was used for $\kappa=1$. Bayesian GOAMP with $20$ iterations is compared to Bayesian GAMP Takeuchi251, BIHT Matsumoto22, and GLasso Plan13Thrampoulidis15 with $20$, $20$, and $50$ iterations, respectively. $10^{4}$ independent trials were simulated for all algorithms.
  • Figure 4: Squared norm for the normalized errors versus $\delta=M/\{k\log(N/k)\}$ for 1-bit compressed sensing with $P/\sigma^{2}=40$ dB in the sublinear sparsity limit. For Bayesian GOAMP, the state evolution recursion (\ref{['moduleA_z_var_bar']}), (\ref{['moduleA_x_var_bar']}), (\ref{['moduleB_x_var_bar_Bayes']}), and (\ref{['moduleB_z_var_bar_Bayes']}) was solved with $50$ iterations. For Bayesian GAMP, state evolution recursion Takeuchi251 with $50$ iterations was evaluated for standard Gaussian sensing matrices. The state evolution results do not depend on $\gamma$.

Theorems & Definitions (18)

  • Definition 1: Pseudo-Lipschitz
  • Definition 2: $\mathrm{PL}(j)$ Convergence
  • Theorem 1
  • Theorem 2
  • Corollary 1
  • Theorem 3
  • Lemma 1
  • Lemma 2: Rangan192Takeuchi20
  • Lemma 3
  • Lemma 4: Takeuchi251
  • ...and 8 more