From sparse recovery to plug-and-play priors, understanding trade-offs for stable recovery with generalized projected gradient descent

TL;DR

This work unifies sparse recovery and deep-prior-based inverse problems under Generalized Projected Gradient Descent (GPGD), extending convergence guarantees to scenarios with model error and approximate projections. It introduces generalized backprojections to handle structured noise (e.g., sparse outliers) and proves a linear convergence/robustness bound governed by a restricted Lipschitz constant and restricted isometry constants under δβ < 1. A Normalize Idempotent Regularization (NIPR) is proposed to control projection errors in deep priors, with empirical evidence showing improved stability and preserved reconstruction quality across denoising, super-resolution, and autoencoder-based priors. The results illuminate trade-offs between identifiability (model fit) and stability under noise and projection imperfections, providing a principled design framework for robust, plug-and-play-like inverse problems. Potential impact includes more reliable deployment of learned priors in imaging applications where noise structures deviate from Gaussian assumptions and projections are imperfect.

Abstract

We consider the problem of recovering an unknown low-dimensional vector from noisy, underdetermined observations. We focus on the Generalized Projected Gradient Descent (GPGD) framework, which unifies traditional sparse recovery methods and modern approaches using learned deep projective priors. We extend previous convergence results to robustness to model and projection errors. We use these theoretical results to explore ways to better control stability and robustness constants. To reduce recovery errors due to measurement noise, we consider generalized back-projection strategies to adapt GPGD to structured noise, such as sparse outliers. To improve the stability of GPGD, we propose a normalized idempotent regularization for the learning of deep projective priors. We provide numerical experiments in the context of sparse recovery and image inverse problems, highlighting the trade-offs between identifiability and stability that can be achieved with such methods.

Figures (11)

• Figure 1: Adaptation to sparse noise: Illustration of the trade-off between stability to noise sparsity and identifiability of $\Sigma$. We show the Normalized error bound for 90$\%$ of the experiments with respect to the sparsity $s$ of outliers. The sparser the noise is, the greater the identifiability.
• Figure 2: Super-resolution of images using GPGD without regularization. (a) represents recovery error for 50 images. The blue curve is associated with the recovery of (b). The quantity $x-\hat{x}$ quickly diverges after reaching the optimal solution and the images become unusable.
• Figure 3: Super-resolution of images using NIPR. (a) represents recovery error for 50 images. The blue curve is associated with the recovery of (b). The quantity $x-\hat{x}$ is slowly diverging after reaching the optimal solution. Yet the obtained images are still recovering the original image correctly.
• Figure 4: Importance of the restricted Lipschitz constant for stable recovery: the case of sparse recovery. Top: Normalized error bound for 95$\%$ of the experiments with respect to the sparsity $k$ of the unknown. Bottom: convergence for one experiment with $k=9$. We observe that worsening the Lipschitz constant (through the parameter $\alpha$) deteriorates both the convergence rate and the identifiability properties of the algorithm.
• Figure 5: Impact of the step size on the stability of PGD for sparse recovery. Top: phase transition diagram for stable recovery. Bottom: impact of $\mu$ on convergence for $k=4$

Theorems & Definitions (12)

• Definition 2.1
• Definition 2.2: Generalized projection
• Definition 2.3: Orthogonal projection
• Definition 2.4: Restricted Lipschitz property
• Theorem 3.1
• proof : Proof of Theorem \ref{['th:stability']}
• Lemma B.1
• proof : Proof of Lemma \ref{['lem:approx_orth_proj']}
• Lemma E.1
• proof : Proof of Lemma \ref{['lem:product_model_proj']}