Aspects of a Generalized Theory of Sparsity based Inference in Linear Inverse Problems
Ryan O'Dowd, Raghu G. Raj, Hrushikesh N. Mhaskar
TL;DR
The paper develops a generalized sparsity framework for linear inverse problems, motivated by Compound Gaussian priors and their empirical success. It introduces a flexible regularizer-based notion of sparsity via $(K, R, \epsilon)$-sparsity and a weak null space property, enabling generalized versions of basis pursuit and IRLS within the CG context. A key contribution is the Generalized IRLS (G-IRLS) algorithm, with convergence guarantees to near-minimizers under a $(K, \gamma, \delta)$-NSP, and an explicit link between CG priors (notably Compound Laplacian) and subadditive, concave regularizers. The framework both unifies and extends classical CS results and CG-based inference, providing a pathway toward a full theory for generalized-sparsity-based CG inference, while highlighting directions for extending to broader CG distributions and convergence-rate analyses.
Abstract
Linear inverse problems are ubiquitous in various science and engineering disciplines. Of particular importance in the past few decades, is the incorporation of sparsity based priors, in particular $\ell_1$ priors, into linear inverse problems, which led to the flowering of fields of compressive sensing (CS) and sparsity based signal processing. More recently, methods based on a Compound Gaussian (CG) prior have been investigated and demonstrate improved results over CS in practice. This paper is the first attempt to identify and elucidate the fundamental structures underlying the success of CG methods by studying CG in the context of a broader framework of generalized-sparsity-based-inference. After defining our notion of generalized sparsity we introduce a weak null space property and proceed to generalize two well-known methods in CS, basis pursuit and iteratively reweighted least squares (IRLS). We show how a subset of CG-induced regularizers fits into this framework.