Direct and Converse Theorems in Estimating Signals with Sublinear Sparsity

TL;DR

The paper analyzes estimation of sublinearly sparse signals over an AWGN channel with noise variance scaling $oldsymbol{ u}^2=rac{oldsymbol{ ext{σ}}^2}{\, ext{log}(N/k)}$, establishing direct and converse limits in the sublinear sparsity regime. It proves achievability via a maximum-likelihood estimator that attains vanishing $k^{-1}\,\|oldsymbol{x}-\hat{oldsymbol{x}}\|_2^2$ when $oldsymbol{σ}^2<u_{ ext{min}}^2/2$, and a converse showing no estimator can beat the prior $k^{-1}\|\boldsymbol{x}\|_2^2$ by more than vanishing amounts when $oldsymbol{σ}^2>u_{ ext{max}}^2/2$, with the two thresholds coinciding for constant-amplitude non-zeros. These results imply asymptotic Bayes-optimality of a separable Bayesian denoiser used in AMP for sublinear sparsity at small $oldsymbol{ abla}$, and are complemented by numerical demonstrations of both non-separable denoisers and AMP performance in compressed sensing. The work highlights that finite-size and non-Gaussian estimation-error effects can limit practical gains from altered denoisers in AMP, pointing to directions for mitigating such effects.

Abstract

This paper addresses the estimation of signals with sublinear sparsity sent over the additive white Gaussian noise channel. This fundamental problem arises in designing denoisers used in message-passing algorithms for sublinear sparsity. The main results are direct and converse theorems in the sublinear sparsity limit, where the signal sparsity grows sublinearly in the signal dimension as the signal dimension tends to infinity. As a direct theorem, the maximum likelihood estimator is proved to achieve vanishing square error in the sublinear sparsity limit if the noise variance is smaller than a threshold. As a converse theorem, all estimators cannot achieve square errors smaller than the signal power under a mild condition if the noise variance is larger than another threshold. In particular, the two thresholds coincide with each other when non-zero signals have constant amplitude. These results imply the asymptotic optimality of an existing separable Bayesian estimator used in approximate message-passing for sublinear sparsity.

Figures (3)

• Figure 1: Square error versus SNR $1/\sigma^{2}$ in dB for $N=2^{16}$, $k=16$, constant non-zero signals $\mathcal{U}=\{1\}$, and the AWGN channel (\ref{['AWGN']}). $10^{5}$ independent trials were simulated for the non-separable Bayesian estimator (\ref{['non_separable_denoiser']}), separable Bayesian estimator Takeuchi251, and ML estimator (\ref{['ML']}). The vertical dashed line shows the threshold $1/\sigma^{2}=2$ in Theorems \ref{['theorem_Gallager']} and \ref{['theorem_converse']}.
• Figure 2: Square errors of AMP using separable and non-separable Bayesian estimators versus $\delta=M/\{k\log(N/k)\}$ for $N=2^{16}$, $k=16$, constant non-zero signals $\mathcal{U}=\{1\}$, $1/\sigma^{2}=40$ dB, $30$ iterations, and the measurement model (\ref{['compressed_sensing']}). According to the numerical simulations shown in Fig. \ref{['fig1']}, in non-separable AMP, the non-separable Bayesian estimator was used when the input SNR to the denoiser is larger than 6 dB. Otherwise, the separable Bayesian estimator was used. $10^{5}$ independent trials were simulated. The optimal threshold Reeves20 is approximately $\delta_{\mathrm{opt}}\approx0.5$ while the threshold for AMP Takeuchi251 is $\delta_{\mathrm{AMP}}\approx 2$.
• Figure 3: Square errors after denoising versus the square error normalized by $N$ before denoising for $N=2^{16}$, $k=16$, constant non-zero signals $\mathcal{U}=\{1\}$, and $1/\sigma^{2}=40$ dB. The measurement model (\ref{['compressed_sensing']}) was postulated for the separable/non-separable AMP while the AWGN channel (\ref{['AWGN']}) was assumed for the separable/non-separable Bayesian estimators. In non-separable AMP, the non-separable Bayesian estimator was used when the input SNR to the denoiser is larger than 6 dB. Otherwise, the separable Bayesian estimator was used. $10^{5}$ independent trials were simulated.

Theorems & Definitions (21)

• Theorem 1: Achievability
• Theorem 2: Converse
• Proposition 1
• Lemma 1
• Proposition 2
• Definition 1
• Lemma 2
• Lemma 3
• Proposition 3
• Proposition 4