Randomized and Inner-product Free Krylov Methods for Large-scale Inverse Problems

TL;DR

The paper tackles large-scale linear inverse problems by replacing traditional inner-product-based Krylov projections with inner-product-free approaches that minimize a residual-like objective. It introduces two new methods, sCMRH and sLSLU, which build a nonorthogonal Hessenberg basis and solve sketched projected least-squares problems, yielding unbiased or near-minimal residual estimates. The methods extend to Tikhonov regularization via sketching and provide theoretical residual-bounding guarantees under randomization. Numerical experiments on deblurring, neutron tomography, and real datasets show that the proposed schemes match or closely follow GMRES/LSQR performance while avoiding inner products, making them attractive for mixed-precision and parallel computing.

Abstract

Iterative Krylov projection methods have become widely used for solving large-scale linear inverse problems. However, methods based on orthogonality include the computation of inner-products, which become costly when the number of iterations is high; are a bottleneck for parallelization; and can cause the algorithms to break down in low precision due to information loss in the projections. Recent works on inner-product free Krylov iterative algorithms alleviate these concerns, but they are quasi-minimal residual rather than minimal residual methods. This is a potential concern for inverse problems where the residual norm provides critical information from the observations via the likelihood function, and we do not have any way of controlling how close the quasi-norm is from the norm we want to minimize. In this work, we introduce a new Krylov method that is both inner-product-free and minimizes a functional that is theoretically closer to the residual norm. The proposed scheme combines an inner-product free Hessenberg projection approach for generating a solution subspace with a randomized sketch-and-solve approach for solving the resulting strongly overdetermined projected least-squares problem. Numerical results show that the proposed algorithm can solve large-scale inverse problems efficiently and without requiring inner-products.

Figures (13)

• Figure 1: Schematic representation of the sketching of a matrix $A$ using a sketch $S$.
• Figure 2: Relative reconstruction error norms (left) and residual norms (right). In this case, sketched CMRH uses the pivots dictated by the maximum absolute value from a set of randomly sampled coefficients ($5$).
• Figure 3: Measured noisy data, and reconstructed images using GMRES, CMRH, and sCMRH.
• Figure 4: Relative reconstruction error norms (left) and residual norms (right). In this case, sketched LSLU uses the pivots dictated by the maximum absolute value from a set of randomly sampled coefficients (25).
• Figure 5: Measured noisy data, true solution, and reconstructed images from LSQR, LSLU, and sLSLU. The image proportions are accurate but, to aid visualization, the relative size between images is not.