Priorconditioned Sparsity-Promoting Projection Methods for Deterministic and Bayesian Linear Inverse Problems

TL;DR

This work addresses the challenge of solving sparsity-promoting linear inverse problems, which are often ill-posed, by introducing priorconditioning into generalized Krylov subspace methods (PS-GKS). PS-GKS accelerates IRLS iterations in both deterministic MM-based and Bayesian GSBL/IAS formulations by transforming the problem into a space where the projected system exhibits clustered spectrum and improved conditioning, enabling faster convergence with smaller subspaces. The authors prove a key sparsity theorem showing eigenvalue clustering in the priorconditioned system, and develop practical variants including restarting and recycling to manage memory and computational costs; they also provide a GPU-accelerated, scalable approach for computing the required pseudoinverses. Numerical experiments in 1D and CT reconstruction demonstrate that PS-GKS outperforms existing S-GKS and related methods, with automatic parameter selection in the IAS setting and robust performance to MM smoothing parameters. Overall, PS-GKS offers a principled, scalable framework for efficient, sparse inverse problem solving with potential applicability to dynamic problems and broader hierarchical priors.

Abstract

High-quality reconstructions of signals and images with sharp edges are needed in a wide range of applications. To overcome the large dimensionality of the parameter space and the complexity of the regularization functional, {sparisty-promoting} techniques for both deterministic and hierarchical Bayesian regularization rely on solving a sequence of high-dimensional iteratively reweighted least squares (IRLS) problems on a lower-dimensional subspace. Generalized Krylov subspace (GKS) methods are a particularly potent class of hybrid Krylov schemes that efficiently solve sequences of IRLS problems by projecting large-scale problems into a relatively small subspace and successively enlarging it. We refer to methods that promote sparsity and use GKS as S-GKS. A disadvantage of S-GKS methods is their slow convergence. In this work, we propose techniques that improve the convergence of S-GKS methods by combining them with priorconditioning, which we refer to as PS-GKS. Specifically, integrating the PS-GKS method into the IAS algorithm allows us to automatically select the shape/rate parameter of the involved generalized gamma hyper-prior, which is often fine-tuned otherwise. Furthermore, we proposed and investigated variations of the proposed PS-GKS method, including restarting and recycling (resPS-GKS and recPS-GKS). These respectively leverage restarted and recycled subspaces to overcome situations when memory limitations of storing the basis vectors are a concern. We provide a thorough theoretical analysis showing the benefits of priorconditioning for sparsity-promoting inverse problems. Numerical experiment are used to illustrate that the proposed PS-GKS method and its variants are competitive with or outperform other existing hybrid Krylov methods.

Figures (10)

• Figure 1: Comparison of GKS (with equal weights), S-GKS, and PS-GKS reconstructions for the 1D cosine problem with MM weights (top row) and IAS weights (bottom row).
• Figure 1: Test 1. The RRE, SSIM, and Gini index for MM (top row) and IAS (bottom row) weights.
• Figure 1: Test 1. Comparison of the basis vectors generated by S-GKS and PS-GKS, with MM weights ($p = 1$, $\varepsilon = 10^{-4}$) and IAS weights ($r = -1$, $\beta = 1$). (left column) select S-GKS basis vectors; (right column) select PS-GKS basis vectors, transformed into the same space as the S-GKS basis vectors to aid the comparison. In all plots, the ground truth vector is overlayed with a dashed black line.
• Figure 1: Test 1. Spectrum of $\mathbf{Q}_{\mu}^{\text{st}}$ and $\mathbf{Q}_{\mu}^{\text{pr}}$ as the PS-GKS iterations progress. We show the theoretical eigenvalue bounds predicted by the theory developed in \ref{['sub:analysis_of_priorconditioning']}, as well as the realized spectra. (top row) Results using MM weights. (bottom row) Results using IAS weights.
• Figure 2: Performance comparison of the GKS (with equal weights), S-GKS (including restarting and recycling), and the proposed PS-GKS methods for the 1D cosine problem with MM (top row) and IAS (bottom row) weights. Reported are the RRE (first column), the Gini index (measuring sparsity) of $\boldsymbol{\Psi} \mathbf{x}$ (second column), and the condition number of the projected least squares problem (third column).

Theorems & Definitions (14)

• Remark 2.1
• Remark 3.1: Restarting and recycling
• Remark 4.1
• Theorem 4.2
• Proof 1
• Theorem 4.3
• Proof 2
• Remark 4.4: Computational costs of PS-GKS
• Remark 4.5: Incorporating $\mathbf{x}_0$ into $\mathcal{V}_0$
• Theorem 4.6