Alternating Subspace Method for Sparse Recovery of Signals

TL;DR

The Alternating Subspace Method is presented, which integrates the principles of the greedy methods and the splitting methods and enhances the splitting method by achieving fidelity in a subspace-restricted fashion, and has the potential to be competitive in different sparse recovery applications.

Abstract

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Numerous renowned algorithms for tackling the compressed sensing problem employ an alternating strategy, which typically involves data matching in one module and denoising in another. We present a novel approach, the Alternating Subspace Method (ASM), which integrates the principles of the greedy methods (e.g., the orthogonal matching pursuit type methods) and the splitting methods (e.g., the approximate message passing type methods). Crucially, ASM enhances the splitting method by achieving fidelity in a subspace-restricted fashion. \textcolor{black}{We reveal that such a restriction strategy guarantees global convergence via proximal residual control and establish its local geometric convergence on the LASSO problem.} Numerical experiments on the LASSO, channel estimation, and dynamic compressed sensing problems demonstrate its high convergence rate and its capacity to incorporate different prior distributions. Overall, the proposed method is promising in terms of efficiency, accuracy, and flexibility, and has the potential to be competitive in different sparse recovery applications.

Figures (6)

• Figure 1: Median convergence behaviors of various algorithms for the LASSO problem with $M=200$, $N=500$, and $\lambda=10^{-3}$, evaluated over $500$ independent realizations. The term "RP" (row projection) denotes the projection of the error $e^k=x^k - x^*$ onto the row space of $A$, which decays rapidly while the null space component is actually the dominant part of $e^k$. The triangle marks the transition point beyond which $|E^k| \leq M$; note the accelerated convergence of ASM near this transition.
• Figure 2: Median relative KKT residual versus iteration count across 200 realizations for different measurement matrices. Here, $A \in \mathbb{R}^{200 \times 400}$ is I.I.D. Gaussian (top) or row-orthogonal (bottom) and SNR $= 30$ dB.
• Figure 3: The median relative KKT residual versus iteration over 200 realizations for different SNR. $A\in\mathbb R^{200\times 400}$ is I.I.D. Gaussian. SNR is $10$dB (Top) or $50$dB (Bottom).
• Figure 4: The median relative KKT residual versus iteration over 200 realizations for different signal scales. I.I.D. Gaussian $A$ satisfies that $A\in\mathbb R^{200\times 800}$ (Top) or $A\in\mathbb R^{200\times 1600}$ (Bottom) and SNR =$30$dB.
• Figure 5: Median NMSE versus iteration count across 200 realizations for different SNR settings: SNR = 10 dB (left) and SNR = 30 dB (right). The channel $h$ is simulated using \ref{['eq:HMC_pri']} with $p_{01} = 1/750$, $p_{10} = 1/250$, and $\sigma_0^2 = (p_{01} + p_{10}) / p_{01}$. Here, $M = 200$ and $N = 400$.

Theorems & Definitions (33)

• Definition 1: Generic Conditions
• Theorem 1: Local geometric convergence
• Proposition 1: Convergence of ADMM in terms of $R^k$
• Theorem 2: Global convergence framework
• Definition 2: Non-Monotonic Descent Condition
• Lemma 1
• Definition 3: Safe Averaging Rule
• Lemma 2
• Proposition 2
• Theorem 3: Global Convergence of ASM-L1