Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noise Variance Estimation Using Asymptotic Residual in Compressed Sensing

Ryo Hayakawa

TL;DR

The paper addresses the challenge of estimating an unknown noise variance $σ_v^2$ in compressed sensing from a single measurement vector. It introduces asymptotic residual matching (ARM), which uses a CGMT-derived asymptotic expression for the ℓ1 residual to recover $σ_v^2$ by aligning it with the empirically observed residual, while also providing a principled initialization and iteration scheme for the regularization parameter $λ$. The method extends to non-sparse structured signals, such as binary vectors via box relaxation, and demonstrates superior variance estimation and downstream reconstruction performance in simulations, particularly for small problem sizes. ARM thus offers a practical, theory-grounded approach to robust reconstruction when noise characteristics are unknown, with potential extensions to broader signal models and matrix ensembles.

Abstract

In compressed sensing, measurements are typically contaminated by additive noise, and therefore, information about the noise variance is often needed to design algorithms. In this paper, we propose a method for estimating the unknown noise variance in compressed sensing problems. The proposed method, called asymptotic residual matching (ARM), estimates the noise variance from a single measurement vector on the basis of the asymptotic result for the $\ell_{1}$ optimization problem. Specifically, we derive the asymptotic residual corresponding to the $\ell_{1}$ optimization and show that it depends on the noise variance. The proposed ARM approach obtains the estimate by comparing the asymptotic residual with the actual one, which can be obtained by empirical reconstruction without the information on the noise variance. For the proposed ARM, we also propose a method to choose a reasonable parameter based on the asymptotic residual. Simulation results show that the proposed noise variance estimation outperforms several conventional methods, especially when the problem size is small. We also show that, by using the proposed method, we can tune the regularization parameter of the $\ell_{1}$ optimization to achieve good reconstruction performance, even when the noise variance is unknown.

Noise Variance Estimation Using Asymptotic Residual in Compressed Sensing

TL;DR

The paper addresses the challenge of estimating an unknown noise variance in compressed sensing from a single measurement vector. It introduces asymptotic residual matching (ARM), which uses a CGMT-derived asymptotic expression for the ℓ1 residual to recover by aligning it with the empirically observed residual, while also providing a principled initialization and iteration scheme for the regularization parameter . The method extends to non-sparse structured signals, such as binary vectors via box relaxation, and demonstrates superior variance estimation and downstream reconstruction performance in simulations, particularly for small problem sizes. ARM thus offers a practical, theory-grounded approach to robust reconstruction when noise characteristics are unknown, with potential extensions to broader signal models and matrix ensembles.

Abstract

In compressed sensing, measurements are typically contaminated by additive noise, and therefore, information about the noise variance is often needed to design algorithms. In this paper, we propose a method for estimating the unknown noise variance in compressed sensing problems. The proposed method, called asymptotic residual matching (ARM), estimates the noise variance from a single measurement vector on the basis of the asymptotic result for the optimization problem. Specifically, we derive the asymptotic residual corresponding to the optimization and show that it depends on the noise variance. The proposed ARM approach obtains the estimate by comparing the asymptotic residual with the actual one, which can be obtained by empirical reconstruction without the information on the noise variance. For the proposed ARM, we also propose a method to choose a reasonable parameter based on the asymptotic residual. Simulation results show that the proposed noise variance estimation outperforms several conventional methods, especially when the problem size is small. We also show that, by using the proposed method, we can tune the regularization parameter of the optimization to achieve good reconstruction performance, even when the noise variance is unknown.

Paper Structure

This paper contains 19 sections, 3 theorems, 44 equations, 16 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Noise Variance Estimation in Compressed Sensing
  3. Related Work
  4. Asymptotic Residual for $\ell_{1}$ Optimization
  5. Proposed Noise Variance Estimation
  6. Asymptotic Residual Matching
  7. Initialization of $\lambda$
  8. Iterative Estimation
  9. Extension to Other Structured Vectors
  10. Simulation Results
  11. Sparse Vector Reconstruction
  12. Binary Vector Reconstruction
  13. Conclusion and Future Work
  14. Proof of Theorem \ref{['th:main']}
  15. CGMT
  16. ...and 4 more sections

Key Result

Theorem 4.1

We assume that Assumption ass:problem is satisfied. We also assume that the optimization problem $\min_{\alpha > 0} \max_{\beta \ge 0} F(\alpha, \beta)$ has a unique optimizer $(\alpha^{\ast}, \beta^{\ast})$The uniqueness can be proven under some conditions. For example, if the set of minimizers of and $X \sim p_{\text{X}}, G \sim p_{\text{G}}$. Then, the asymptotic value of the objective functio

Figures (16)

  • Figure 1: Objective function and residual for the optimizer ($N = 100$, $M = 90$, $p_{0} = 0.8$, $\lambda = 0.001$, $p_{\text{X}}(x)$: Bernoulli-Gaussian distribution).
  • Figure 2: Asymptotic residual of the $\ell_{1}$ optimization ($\Delta = 0.8$, $p_{0} = 0.9$).
  • Figure 3: $\sigma_{\text{v}}^{2}=0.01$
  • Figure 4: $\sigma_{\text{v}}^{2}=0.001$
  • Figure 5: $\sigma_{\text{v}}^{2}=0.0001$
  • ...and 11 more figures

Theorems & Definitions (5)

  • Theorem 4.1
  • proof
  • Remark 5.1: Advantage of Using Residual of $\ell_{1}$ Optimization
  • Corollary 6.1
  • Theorem A.1: CGMT