Flexible inner-product free Krylov methods for inverse problems

TL;DR

This work addresses large-scale, ill-posed inverse problems that incorporate variational regularization beyond the standard two-norm. It develops flexible, inner-product free Krylov solvers, including flexible Hessenberg and a new generalized variant, equipped with iteration-dependent right preconditioning and augmented with randomized sketch-and-solve formulations. The contributions include the introduction of FCMRH, FLSLU, hybrid and iteratively-reweighted variants, and multiple sketching-based extensions, along with theoretical monotonicity considerations and extensive numerical demonstrations on deblurring and CT problems with $\ell_1$ and total-variation regularizers. The proposed methods offer memory- and communication-efficient alternatives that remain competitive in accuracy, especially in low-precision settings, and enable parallelizable computations crucial for large-scale inverse problems.

Abstract

Flexible Krylov methods are a common standpoint for inverse problems. In particular, they are used to address the challenges associated with explicit variational regularization when it goes beyond the two-norm, for example involving an $\ell_p$ norm for $0 < p \leq 1$. Moreover, inner-product free Krylov methods have been revisited in the context of ill-posed problems, to speed up computations and improve memory requirements by means of using low precision arithmetics. However, these are effectively quasi-minimal residual methods, and can be used in combination with tools from randomized numerical linear algebra to improve the quality of the results. This work presents new flexible and inner-product free Krylov methods, including a new flexible generalized Hessenberg method for iteration-dependent preconditioning. Moreover, it introduces new randomized versions of the methods, based on the sketch-and-solve framework. Theoretical considerations are given, and numerical experiments are provided for different variational regularization terms to show the performance of the new methods.

Figures (11)

• Figure 1: Example 2. True solution modeling a starry night and noisy measurements simulating atmospheric blur.
• Figure 2: Example 2. Relative error norms for different methods without explicit regularization (left) and for a fixed regularization parameter that is good for the full dimensional problem (right).
• Figure 3: Example 1. Relative error norms for different regularization parameter choices (each column), and for different methods. Note that both FISTA and SpaRSA require a regularization parameter to be chosen before the iterations, so we use the one computed by H-FGMRES with the discrepancy principle at the end of the iterations. Note that the labels are only displayed once for each row.
• Figure 4: Example 2. True solution and noisy measurements, also known as sinogram.
• Figure 5: Example 2. Relative error norms for different methods based on (flexible) GK and (flexible) generalized Hessenberg.

Theorems & Definitions (14)

• Lemma 1
• proof
• Proposition 1
• proof
• Corollary 1
• proof
• Proposition 2
• proof
• Corollary 2
• proof