Optimization landscape of $\ell_0$-Bregman relaxations
Jonathan Chirinos-Rodriguez, Cédric Févotte, Emmanuel Soubies
TL;DR
This work tackles sparse recovery in noisy linear/inverse problems with non-quadratic data fidelity by introducing the $ ext{ell}_0$-Bregman relaxation (B-rex) and the BRSC property. The approach generalizes prior LRIP/RSC analyses to non-quadratic settings, enabling conditions under which BRSC guarantees that the relaxed problem $J_\Psi$ has an isolated sparsest critical point that coincides with the global minimizer of the original $J_0$. The authors establish an oracle-recovery framework: under suitable BRSC constants and a prescribed interval for the regularization parameter $\lambda_0$, the oracle solution is the unique global minimizer and sparsity-isolated. They specialize the theory to Gaussian LS and Poisson KL fidelities, obtaining improved LS bounds and a novel KL-based result, with concrete corollaries for both $\Psi=\ell_2$ and KL-Bregman generators. The results offer a principled route to exact sparse recovery in broader data-fidelity contexts and provide practical bounds for parameter selection in Gaussian and Poisson regression problems.
Abstract
In this paper, we study (noisy) linear systems, and their $\ell_0$-regularized optimization problems, coupled with general data fidelity terms. Recent approaches for solving this class of problems have proposed to consider non-convex exact continuous relaxations that preserve global minimizers while reducing the number of local minimizers. Within this framework, we consider the class of $\ell_0$-Bregman relaxations, and establish sufficient conditions under which a critical point is isolated in terms of sparsity, in the sense that any other critical point has a strictly larger cardinality. In this way, we ensure a form of uniqueness in the solution structure. Furthermore, we analyze the exact recovery properties of such exact relaxations. To that end, we derive conditions under which the oracle solution (i.e., the one sharing the same support as the ground-truth) is the unique global minimizer of the relaxed problem, and is isolated in terms of sparsity. Our analysis is primarily built upon a novel property we introduce, termed the Bregman Restricted Strong Convexity. Finally, we specialize our general results to both sparse Gaussian (least-squares) and Poisson ((generalized) Kullback-Leibler divergence) regression problems. In particular, we show that our general analysis sharpens existing bounds for the LS setting, while providing an entirely new result for the KL case.