Compressed sensing with l0-norm: statistical physics analysis and algorithms for signal recovery

TL;DR

This work tackles noiseless compressed sensing with an $\ell_0$-norm penalty to recover a sparse signal from underdetermined measurements. It employs the replica-method-based statistical physics framework to reveal a clustered, 1RSB landscape of minimizers and derives two AMP-family algorithms (ASP and ASP_o) that exploit this structure. The authors compute a phase diagram with an algorithmic threshold $\alpha_{c,\mathrm{1RSB}}(\rho_o)$ that delineates regimes of exact recovery and partial recovery, showing that the nonconvex $\ell_0$ approach can approach Bayesian-optimal performance in practice. The proposed ASP/ASP_o algorithms, tracked by state evolution, outperform LASSO in the recoverable regime, providing a computationally efficient route to high-sparsity signal recovery and motivating extensions to noisy measurements and broader inference problems.

Abstract

Noiseless compressive sensing is a protocol that enables undersampling and later recovery of a signal without loss of information. This compression is possible because the signal is usually sufficiently sparse in a given basis. Currently, the algorithm offering the best tradeoff between compression rate, robustness, and speed for compressive sensing is the LASSO (l1-norm bias) algorithm. However, many studies have pointed out the possibility that the implementation of lp-norms biases, with p smaller than one, could give better performance while sacrificing convexity. In this work, we focus specifically on the extreme case of the l0-based reconstruction, a task that is complicated by the discontinuity of the loss. In the first part of the paper, we describe via statistical physics methods, and in particular the replica method, how the solutions to this optimization problem are arranged in a clustered structure. We observe two distinct regimes: one at low compression rate where the signal can be recovered exactly, and one at high compression rate where the signal cannot be recovered accurately. In the second part, we present two message-passing algorithms based on our first results for the l0-norm optimization problem. The proposed algorithms are able to recover the signal at compression rates higher than the ones achieved by LASSO while being computationally efficient.

Figures (3)

• Figure 1: (Top) Phase diagram for perfect recovery in compressed sensing. The $\ell_1$, and BO lines give the algorithmic thresholds for $\ell_1$ based reconstruction and Bayesian optimal AMP respectively. Above those lines, the corresponding algorithms succeed with high probability for large systems. The $\ell_0$ 1RSB line instead, gives the recovery threshold for our algorithms as predicted by our replica analysis. Lastly, the $\ell_0$ IT is the ultimate limit for any algorithmic able to perform global optimization of the $\ell_0$-based cost. Since it matches the diagonal, in principle $\ell_0$-based reconstruction saturates the information-theoretic bounds. In the region between $\ell_0$ IT and the $\ell_0$ 1RSB though, it is algorithmically hard to find the global optimum of the loss. (Bottom) We plot $\left\langle\mathcal{L}(\bm{x},\bm{y},\bm{F},\lambda) \right\rangle /{\lambda N}=e/\lambda+\rho$ as a function of $\lambda$, for $\rho_o=0.6$ and several values of $\alpha$. We show the predictions from the RS Informed saddle point and the 1RSB Uninformed one. In the regime when the signal recovery is easy ($\alpha>0.83$) both saddle points describe the true signal as $\lambda$ goes to zero, we thus have $e/\lambda+\rho\to \rho_o$. At lower $\alpha$ instead, the uninformed saddle point no longer describes the true signal, but configurations at larger density.
• Figure 2: Stability conditions for the 1RSB saddle point both in the regimes where it yields signal recovery ($\alpha=0.9$) and where it does not ($\alpha=0.7$). Regardless of the regime, we observe that the 1RSB saddle point is stable when $\lambda$ is low enough, i.e. the curves lay above zero.
• Figure 3: (Top) The overlap with the signal when running the ${\mathrm{ASP}}$ (orange), ${\mathrm{ASP}_o}$ (blue) and LASSO (red) algorithms at different values of the regularization prefactor $\lambda$. The dashed lines correspond to finite size simulations while the full lines correspond to the infinite size prediction from State Evolution (SE). For this experiment, we set $\rho_o=0.6 and \alpha=0.87$. The ${\mathrm{ASP}_o}$ parameter is fixed to $\xi=0.7$. (Bottom) The quantity $V$ (for ${\mathrm{ASP}_o}$ and LASSO) and $V_1+s V_0$ (${\mathrm{ASP}}$) in the same experimental setting.