Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing

TL;DR

This work targets sparse signal reconstruction with the SCAD penalty by developing a one-step replica-symmetry-breaking (1RSB) extension of approximate message passing (AMP). It derives explicit 1RSB-AMP updates and 1RSB state evolution (SE), showing macroscopic agreement with RS results while expanding the algorithmic reconstruction limit via a Parisi-parameter optimization (NCC). The authors analyze fixed points and phase diagrams to identify success, failure, and diverging regimes, demonstrating that 1RSB reduces the diverging region and yields near Bayes-optimal performance, albeit not surpassing it. Numerical experiments corroborate the theory, and the work discusses the thermodynamic quantities governing the 1RSB phase, while outlining avenues for higher-step RSB extensions.

Abstract

We consider sparse signal reconstruction via minimization of the smoothly clipped absolute deviation (SCAD) penalty, and develop one-step replica-symmetry-breaking (1RSB) extensions of approximate message passing (AMP), termed 1RSB-AMP. Starting from the 1RSB formulation of belief propagation, we derive explicit update rules of 1RSB-AMP together with the corresponding state evolution (1RSB-SE) equations. A detailed comparison shows that 1RSB-AMP and 1RSB-SE agree remarkably well at the macroscopic level, even in parameter regions where replica-symmetric (RS) AMP, termed RS-AMP, diverges and where the 1RSB description itself is not expected to be thermodynamically exact. Fixed-point analysis of 1RSB-SE reveals a phase diagram consisting of success, failure, and diverging phases, as in the RS case. However, the diverging-region boundary now depends on the Parisi parameter due to the 1RSB ansatz, and we propose a new criterion -- minimizing the size of the diverging region -- rather than the conventional zero-complexity condition, to determine its value. Combining this criterion with the nonconvexity-control (NCC) protocol proposed in a previous RS study improves the algorithmic limit of perfect reconstruction compared with RS-AMP. Numerical solutions of 1RSB-SE and experiments with 1RSB-AMP confirm that this improved limit is achieved in practice, though the gain is modest and remains slightly inferior to the Bayes-optimal threshold. We also report the behavior of thermodynamic quantities -- overlaps, free entropy, complexity, and the non-self-averaging susceptibility -- that characterize the 1RSB phase in this problem.

Figures (10)

• Figure 1: The blue solid lines represent $\hat{x}$ (left) and $\chi$ (right) plotted against $h$ at $(\Gamma,a,\lambda)=(1,3,1)$. The black dotted lines are for the case of ordinary least square (OLS), and the red broken ones are for the $\ell_1$ minimization or LASSO. These cases are derived as specific limits of the SCAD estimator: OLS and LASSO correspond to the $\lambda=0$ and $a\to \infty$ cases, respectively.
• Figure 2: (Left) Phase diagram on the $\lambda$--$\rho$ plane at $a=3,\alpha=0.5,\sigma_x^2=1$. The horizontal dashed line indicates the limit achievable by NCC, while the dotted–dashed line marks the boundary of the region where SE flow divergence occurs: On the left side of this line, a diverging state appears in the $\chi$--$\epsilon$ plane, whereas it does not exist on the right side. (Right) The reconstruction limit on $\rho$--$\alpha$ plane by NCC (purple solid line) and the principle limit at $(a,\lambda)=(3,0.1)$ (green dashed line) that describes the boundary below which the perfect reconstruction solution becomes unstable. For comparison, the algorithmic (BO, spinodal) and principle (BO, principle) limits of the BO method are also shown. Both panels are reproductions of the corresponding figures from sakata2021scad.
• Figure 3: The AMP trajectory and the SE snapshots on $\chi$--$\epsilon$ plane during NCC. This trajectory is for one realization of $\bm{x} _0$ and $\bm{A}$, and the parameters are $(N,a,\sigma_x^2,\alpha,\rho)=(10^5,3,1,0.5,0.28)$. The initial condition is set to be $\bm{x} ^{(0)}=\bm{0} ,\chi_i^{(0)}=\rho \sigma_x^2$, implying that the corresponding initial condition of SE is $\chi^{(0)}=\epsilon^{(0)}=\rho \sigma_x^2$. The parameter $\lambda$ is initialized at $\lambda=1$ and gradually decreased. Adapted from sakata2021scad with minor modifications.
• Figure 4: Consistency check between 1RSB-SE and 1RSB-AMP for $(\alpha,\rho)=(0.5,0.1)$ at $(a,\lambda,\ell)=(3,1,1)$ (success phase). The curves show the iteration dependence of $\epsilon_{q_0}$ and $\chi$ (left) and the overlaps (right). The AMP result (plus marker) is obtained by the average over 100 different realizations of $(\bm{A} ,\bm{x} _0)$, with standard error bars included. The agreement demonstrates that 1RSB-SE correctly describes the macroscopic dynamics of 1RSB-AMP in this phase.
• Figure 5: The counterpart of Fig. \ref{['fig:se_amp_case1']} for $(\alpha,\rho)=(0.5,0.4)$ and $(a,\lambda,\ell)=(3,0.8,1)$ (failure phase). Again, the 1RSB-AMP trajectories agree well with that by 1RSB-SE, demonstrating the consistency between 1RSB-AMP and 1RSB-SE.