Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inner Product Free Krylov Methods for Large-Scale Inverse Problems

Ariana N. Brown, Julianne Chung, James G. Nagy, Malena Sabaté Landman

TL;DR

This work tackles large-scale rectangular inverse problems by introducing two inner-product free Krylov methods: LSLU, an extension of the Hessenberg-based CMRH to rectangular matrices, and Hybrid LSLU, a regularized projection variant. Both methods operate without inner products, leveraging the Hessenberg process and two Krylov subspaces tied to $A^T A$ and $AA^T$, with theoretical residual bounds linking them to LSQR. Regularization is integrated in the projected space via a tunable parameter $\lambda_k$, selected with (weighted) generalized cross-validation on the projected problem, and a stopping criterion is derived from a projected GCV-like metric. Numerical experiments on IR Tools problems (PRtomo, PRspherical, PRseismic) show Hybrid LSLU achieving performance comparable to Hybrid LSQR, while the inner-product free formulation enables efficient mixing of precision and parallel computation, and the low-rank Hessenberg basis supports scalable uncertainty quantification through posterior covariance approximations.

Abstract

In this study, we introduce two new Krylov subspace methods for solving rectangular large-scale linear inverse problems. The first approach is a modification of the Hessenberg iterative algorithm that is based off an LU factorization and is therefore referred to as the least squares LU (LSLU) method. The second approach incorporates Tikhonov regularization in an efficient manner; we call this the Hybrid LSLU method. Both methods are inner-product free, making them advantageous for high performance computing and mixed precision arithmetic. Theoretical findings and numerical results show that Hybrid LSLU can be effective in solving large-scale inverse problems and has comparable performance with existing iterative projection methods.

Inner Product Free Krylov Methods for Large-Scale Inverse Problems

TL;DR

This work tackles large-scale rectangular inverse problems by introducing two inner-product free Krylov methods: LSLU, an extension of the Hessenberg-based CMRH to rectangular matrices, and Hybrid LSLU, a regularized projection variant. Both methods operate without inner products, leveraging the Hessenberg process and two Krylov subspaces tied to and , with theoretical residual bounds linking them to LSQR. Regularization is integrated in the projected space via a tunable parameter , selected with (weighted) generalized cross-validation on the projected problem, and a stopping criterion is derived from a projected GCV-like metric. Numerical experiments on IR Tools problems (PRtomo, PRspherical, PRseismic) show Hybrid LSLU achieving performance comparable to Hybrid LSQR, while the inner-product free formulation enables efficient mixing of precision and parallel computation, and the low-rank Hessenberg basis supports scalable uncertainty quantification through posterior covariance approximations.

Abstract

In this study, we introduce two new Krylov subspace methods for solving rectangular large-scale linear inverse problems. The first approach is a modification of the Hessenberg iterative algorithm that is based off an LU factorization and is therefore referred to as the least squares LU (LSLU) method. The second approach incorporates Tikhonov regularization in an efficient manner; we call this the Hybrid LSLU method. Both methods are inner-product free, making them advantageous for high performance computing and mixed precision arithmetic. Theoretical findings and numerical results show that Hybrid LSLU can be effective in solving large-scale inverse problems and has comparable performance with existing iterative projection methods.
Paper Structure (15 sections, 2 theorems, 67 equations, 7 figures, 1 table, 5 algorithms)

This paper contains 15 sections, 2 theorems, 67 equations, 7 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Main Contributions
  3. Least Squares with LU Factorization
  4. The Hessenberg Process and CMRH
  5. Extension of the Hessenberg Process for rectangular systems and LSLU
  6. Theoretical bounds for the residual norm of LSLU
  7. Hybrid LSLU
  8. Theoretical bounds for the residual norms for Hybrid LSLU
  9. Computational Considerations
  10. Selecting Regularization Parameters
  11. Stopping Criterion
  12. Numerical Results
  13. Regularization and reconstruction performance
  14. Low-rank approximation for uncertainty quantification
  15. Conclusions

Key Result

Theorem 1

Let $r_k^{QR}$ and $r_k^{LU}$ be the LSQR and LSLU residuals at the kth iteration beginning with the same initial guess $x_0 = 0$, respectively. Then where $\kappa (\hat{R}_{k+1}) = \|\hat{R}_{k+1}\| \|\hat{R}_{k+1}^{-1}\|$ is the condition number of $\hat{R}_{k+1}$.

Figures (7)

  • Figure 1: Relative reconstruction error norms per iteration for LSLU with pivoting using the infinity norm, compared to using the estimated infinity norm as the maximum from a set of randomly sampled coefficients (denoted 'LSLU inf est'). Results for LSQR are provided for reference.
  • Figure 1: Residual norms per iteration for Hybrid LSLU, as well as corresponding bounds from \ref{['thm:bounds']}. Note that the lower bound corresponds to Hybrid LSQR residual norms.
  • Figure 1: Measured noisy data, $b$ (top row) and reconstructed images using Hybrid LSLU (bottom row). The image proportions are accurate but, to aid visualization, the relative size between images is not.
  • Figure 2: Relative reconstruction error norms per iteration of Hybrid LSLU with wGCV and the optimal regularization parameter. The automatically selected stopping iteration is highlighted with a star. Results for Hybrid LSQR with wGCV are provided for reference.
  • Figure 3: Basis vectors for the Krylov subspace (\ref{['eq:K1']}) generated by LSLU and LSQR at iterations $k=2,4,6,8,10$ for the PRseismic example.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Proof 1
  • Theorem 1
  • Proof 2