Truncated Huber Penalty for Sparse Signal Recovery with Convergence Analysis

TL;DR

The paper tackles sparse signal recovery from under-determined measurements by proposing a truncated Huber (TH) penalty φμ with φμ(x)=min(1, x^2/μ^2) to bridge unbiased sparse recovery and differentiable optimization. It develops a TH-penalized optimization framework for both noiseless and noisy settings and constructs a block coordinate descent algorithm based on a surrogate Qμ(x, ω) that reduces to per-coordinate proximal steps, with a μ-continuation scheme to ensure numerical stability. Theoretical contributions include showing that any s-sparse solution recoverable by conventional penalties remains a local optimum under TH for suitable μ, differentiability at optima, and finite-step convergence under spark conditions, plus an extension to gradient-domain denoising. Empirically, TH demonstrates superior robustness and accuracy across synthetic sparse recovery tasks and image denoising, including precise recovery of high-dynamic-range coefficients and edge-preserving gradient-domain performance, highlighting its practical impact for sparse recovery and image processing.

Abstract

Sparse signal recovery from under-determined systems presents significant challenges when using conventional L_0 and L_1 penalties, primarily due to computational complexity and estimation bias. This paper introduces a truncated Huber penalty, a non-convex metric that effectively bridges the gap between unbiased sparse recovery and differentiable optimization. The proposed penalty applies quadratic regularization to small entries while truncating large magnitudes, avoiding non-differentiable points at optima. Theoretical analysis demonstrates that, for an appropriately chosen threshold, any s-sparse solution recoverable via conventional penalties remains a local optimum under the truncated Huber function. This property allows the exact and robust recovery theories developed for other penalty regularization functions to be directly extended to the truncated Huber function. To solve the optimization problem, we develop a block coordinate descent (BCD) algorithm with finite-step convergence guarantees under spark conditions. Numerical experiments are conducted to validate the effectiveness and robustness of the proposed approach. Furthermore, we extend the truncated Huber-penalized model to the gradient domain, illustrating its applicability in signal denoising and image smoothing.

Figures (8)

• Figure 1: Different scalar penalty functions.
• Figure 1: Left: the evolution of both the parameter $\mu$ and objective function values versus the number of iterations in the matrix $A_1\in\mathbb{R}^{64\times512}$ ($r=0.8,s=12$). Right: the evolution of RRE versus the number of iterations.
• Figure 2: Reconstruction results for a decaying signal using $A_2\in\mathbb{R}^{64\times1024}$ (with $F=5$). Top: Noise-free reconstruction; Bottom: Reconstruction with slight noise ($\sigma=0.0001$).
• Figure 3: Comparison of sparse reconstruction performance in the noise-free case using matrices $A_1$ (with $r=0.8$; top row) and $A_2$ (with $F=10$; bottom row). Metrics include success rates (left column), model failures (middle column), and algorithm failures (right column), evaluated for the following regularizers: $L_1$, $TL_1$, $L_1\hbox{-}L_2$, $L_0$, $MCP$, $L_p$, and $TH$.
• Figure 4: Computation time comparison for sparse vector recovery under noise-free conditions using matrices $A_1$ ($r = 0.8$; left) and $A_2$ ($F = 10$; right). Time values (in seconds) are averaged over 50 trials.

Theorems & Definitions (22)

• Lemma 1
• Theorem 2
• Proof 1
• Theorem 3
• Proof 2
• Lemma 4
• Proof 3
• Theorem 5
• Proof 4
• Theorem 6